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Preface 



"Health is a state of complete physical, mental and social well-being and not 
merely the absence of disease or infirmity " 

—Constitution of the World Health Organization 

The National Institute for Occupational Safety and Health (NIOSH) Health Hazard 
Evaluation (HHE) Program was derived from a legislative promise to workers and em- 
ployers that NIOSH would provide timely information regarding whether concentrations 
of substances normally found in the workplace have potentially toxic effects. NIOSH has 
the authority, in response to written request, to enter a workplace and measure potentially 
hazardous exposures, conduct medical evaluation of employees, and obtain available in- 
formation from employers. Though these are essentially the same authorities provided for 
Occupational Safety and Heahh Administration (OSHA) inspections, invoked by the 
same parties who can trigger such inspections, health hazard evaluations (HHEs) are sup- 
posed to be something different. OSHA inspections are intended to ensure compliance 
with existing safety regulations. HHEs arc conducted to identify health hazards in a 
workplace. HHEs emphasize health effects identification through medical investigation 
and exposure assessment techniques combined with an epidemiological perspective, and 
can be at the cutting edge in identifying emerging occupational health hazards through 
scientifically documenting observations of those at the workplace. 

At the request of NIOSH, the Committee to Review the NIOSH HHE Program was 
formed to evaluate the relevance, impact, and future directions of the program. The HHE 
Program is not a traditional research program, but responds to a wide variety of requests 
for assistance to identify a broad spectrum of hazards. The committee's composition re- 
flects this diversity with expertise in respiratory toxicology, public health, occupational 
health, industrial hygiene, risk communication, occupational medicine, occupational epi- 
demiology, indoor environmental quality, and environmental health. The committee also 
has expertise in public health program evaluation (see Appendix E). It was guided, but 
not bound, by the evaluation criteria established by the National Academies Committee 
to Review NIOSH Research Programs. In this evaluation, the committee provides rec- 
ommendations to improve what it believes is already a worthwhile program and provides 
a vision for the HHE Program of the future. 

The committee thanks NIOSH staff for their kind assistance and willingness to share 
their knowledge. Raymond Sinclair, Teresa Schnorr and Allison Tepper were indispensi- 
ble in their efforts to answer the innumerable questions from the committee. NIOSH 
briefers at the committee's September 2007 meeting were very helpful, and included 
Bruce Bernard, Kay Kreiss, Teresa Schnon'. Teresa Seitz, Allison Tepper, Ken Walling- 
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ford, and Lewis Wade. Other NIOSH staff attended the meeting and contributed to the 
discussions including Marlene Ackman, Fred Blosser, Christine Branche, Chad Dowell, 
Kelly Durst, Lynn Jenkins, Greg Lotz, Michelle Martin, Ken Martinez, John Piacentino, 
Doug Trout, Doug Weissman, and Ainsley Weston. Eric Landree and Valerie Williams 
from the RAND Corporation also spoke with the committee. 

The committee received thoughtful input from a large number of stakeholders to in- 
form its deliberations. Members particularly appreciated the willingness of several stake- 
holders to speak directly with the committee diring its meetings. Shelley Davis (Farm- 
worker Justice, Washington, D.C.), Janie Gittleman (Center to Protect Workers' Rights, 
Silver Spring, Maryland), Jose Oliva (Intetfaith Worker Justice, Chicago, Illinois), Frank 
Renshaw (Rohm and Haas Company. Cor>'don, Pennsylvania), Marthe Kent (OSHA Re- 
gion I, Boston, Massachusetts), Andrea Kidd-Taylor (Community Health and Policy, 
Morgan State University, Baltimore), Kenneth Rosenman (Occupational and Environ- 
mental Health, Michigan State University, Lansing), and Joshua Sharfstein (Baltimore 
City Health Department) participated in panel discussions during the committee's second 
meeting. At the committee's third meeting, Linda Ayala discussed (via telecom) her ex- 
periences related to an HHE conducted for the Alameda County (CA) Public Authority 
for In-Home Services; Peggy Hoffman, Glenn Jones, and Barbara Smisko discussed (via 
telecom) experiences regarding HHEs conducted at Kaiser Permanente in California; 
Tom Tripp and Bryant Hardy spoke with the committee (via telecom) about their HHE 
experiences at U.S. Magnesium in Utah; and Barbara Matema and Roger Speakman de- 
scribed an evaluation at a flavor and fragrance manufacturing company in California. 
These discussions were vital to the evaluation process, and provided the committee mul- 
tiple viewpoints. The committee is also grateful to the nearly 60 respondents to its online 
request for information (see Appendix C). Many thoughtful comments about the HHE 
Program were received and provided necessary fuel for discussion. Those comments and 
recommendations are summarized in Appendix D. 

The committee was sorry to lose the services of John Froincs (Center for Occupa- 
tional and Environmental Health, University of California, Los Angeles) as a member 
midway though the evaluation process, but is indebted to him for his thoughtful com- 
ments on early drafts of the manuscript provided as an unpaid consultant. 

This report has been reviewed in draft fonn by individuals chosen for their diverse 
perspectives and technical expertise, in accordance with procedures approved by the Na- 
tional Research Council's Report Review Committee. The purpose of this independent 
review is to provide candid and critical comments that will assist the institution in making 
its published report as sound as possible and to ensure that the report meets institutional 
standards for objectivity, evidence, and responsiveness to the study charge. The review 
comments and draft manuscript remain confidential to protect the integrity of the delib- 
erative process. We thank the following individuals for their participation in the review of 
this report: 

Susan E. Cozzens, Technology Policy and Assessment Center, Georgia Institute of 

Technology 
James W. Curran, Rollins School of Public Health, Emory University 
Edward A. Emmet(, School of Medicine, University of Pennsylvania 
Lora E. Fleming, Miller School of Medicine and Rosenstiel School of Marine and At- 
mospheric Sciences, Miami 
Christopher C. Green, School of Medicine, Wayne State University 
William H. Farland, Vice President for Research, Colorado State University 
Dan Hair, Workers Compensation Fund of Utah 
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Leslie Israel, Center for Occupational and Environmental Health, University of Califor- 
nia, Irvine 

Philip J. Landrigan, Department of Community & Preventive Medicine and Children's 
Environmental Health Center, Mount Sinai School of Medicine 

Scott Schneider, Laborers' Health and Safety Fund of North America, Washington, D.C. 

Mark J. Utell, School of Medicine and Dentistry, University of Rochester 

Eric Welch, Public Administration, University of Illinois at Chicago 

Although the reviewers listed above have provided many constructive comments and 
suggestions, they were not asked to endorse the conclusions or recommendations nor did 
they see the final draft of the report before its release. The review of this report was 
overseen by John C. Bailar, The University of Chicago (Emeritus), and Linda Hawes 
Clever, California Pacific Medical Center. Appointed by the National Research Council, 
they were responsible for making certain that an independent exammation of this report 
was carried out in accordance with institutional procedures and that all review comments 
were carefully considered. Responsibility for the fmal content of this report rests entirely 
with the authoring committee and the institution. 

Finally, the committee wishes to thank National Academies staff Sammantha 
Magsino, program officer, Susan McCutchen, senior program associate, and Tonya Fong 
Yee, senior project assistant, for the many long hours spent guiding this report through 
the committee process. The report could not have come to completion without their 
persistent efforts. 

Rogene F. Henderson 

Chair 
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Summary 



ABSTRACT In 2005. the Nalional Imtiiute for Occupational Safety and Health 
(NIOSH) asked the National Academies to evaluate the relevance, impact, and 
future directions of a series of up to 15 of its research programs, including the 
Health Hazard Evaluation (HHE) Program. The HHE Program does not conduct 
traditional research but is mandated to respond to requests for assistance in iden- 
tifying specific workplace conditions that pose health hazards to workers. In 
2007, the Committee to Review the NIOSH Health Hazard Evaluation Program 
was formed. The committee finds the HHE Program to he highly relevant to and 
to have a major impact on improving occupational health, and it believes that the 
program should be allowed to continue to provide its services and be expanded as 
resources become available. 

In addition to its work to improve workplace conditions, the HHE Program 
responds well to emergencies, such as the terrorist attack on 9/1 1 and disasters 
related to hurricanes. Program staff are uniquely qualified to serve in leadership 
roles under such conditions. Training programs conducted by the HHE Program 
offer excellent opportunities for increasing expertise in occupational health, and 
the committee recommends more extensive tracking ofahmni so that they can be 
called on for their expertise, assistance in identifying emerging issues, and dur- 
ing emergency respon.se. 

The con.sensus of the committee is that the HHE Program is highly effective in 
investigating and advising workplaces when requested. The program fills a spe- 
cial need in the occupational health community by investigating unexpected or 
underestimated workplace hazards and relating (hem to worker exposures or cir- 
cumstances. However, certain occupational groups might not fully benefit from 
program activities, particularly those from small businesses and underserved 
populations. The committee recommends the HHE Program take steps to acquaint 
such groups with its services and elicit more requests for investigations from 
them. In addition, funding limitations and obligations associated with emergency 
response might dilute program efforts and reduce effectiveness. Finallv, the com- 
mittee believes that the HHE Program could help to develop a national occupa- 
tional health surveillance system to facilitate recognition of emerging hazards. 

On the basis of a scoring system of I to 5. with 5 being the highest, the com- 
mittee rates both the relevance and the impact of the HHE Program as 4. If the 
committee had not been restricted to the use of integers, both scores would hcn'c 
been between 4 and 5. 
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2 THE HEALTH HAZARD EVALUATION PROGRAM AT NIOSI I 

OVERVIEW 

In 2005, the National Institute for Occupational Safety and Health (NIOSH) 
asked the National Academies to evaluate the relevance, impact, and future direc- 
tions of up to 15 of its research programs. One of the programs was the Health 
Hazard Evaluation (HHE) Program, which does not conduct traditional research 
but is mandated to respond to requests for assistance to identify specific work- 
place conditions that pose health hazards to workers. In 2007, the National Re- 
search Council formed the Committee to Review the NIOSH Health Hazard 
Evaluation Program. 

The mission of the HHE Program is to respond to written requests to investi- 
gate potential occupational health hazards in workplaces, as defined by the Occu- 
pational Safety and Health Act of 1970 and the Federal Mine and Safety Act of 
1977, and in federal agencies, including the military. The law defines who may 
submit requests for investigations: a request must be from an employer, a union, 
an employee representing at least two other employees, a single employee if the 
work area of concern has three or fewer employees, a federal agency health and 
safety committee or federal employees not covered by such a committee, or the 
secretary of labor (NIOSH, 2007b). Responses to requests vary from written or 
oral consultations on technical matters to full-scale onsitc investigations. The pro- 
gram conducts field evaluations and consultations, responds to emergencies, and 
provides occupational health training for health professionals. 

The committee had several discussions about how to evaluate the relevance of 
the HHE Program ("Does the program address the right issues?") separately from 
its impact ("Docs the program address the issues effectively?"). The committee 
evaluated the relevance of the program in terms of program inputs, activities, and 
outputs, as outlined in the logic model used by NIOSH to summarize the program. 
Program inputs evaluated include strategic planning and use of resources such as 
funding and staffing. Activities evaluated include the marketing of program ser- 
vices to all parts of the workforce, including small businesses and underserved 
populations, and the setting of program priorities, through a triage process, for 
allocation of resources in response to requests for investigations. HHE Program 
efforts to identify emerging hazards are also evaluated. Program outputs evaluated 
for relevance include the formal and informal responses by the HHE Program to 
requests for assistance. The committee also evaluated HHE Program emergency 
response activities. 

The impacts to be evaluated include the outcomes as listed in the HHE Pro- 
gram logic model, such as the reduction in worker risk and the prevention of oc- 
cupational illness, the transfer of program-generated information to relevant em- 
ployers and employees beyond the investigated workplaces, the influence of the 
HHE Program on NIOSH research and policy development, and the effect of the 
program on the general occupational health community, including activities of 
regulatory agencies, organizations of occupafional safety and health professionals, 
and state and local health agencies. 
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Overall, the committee considers the program to be highly relevant and to 
have a major impact on improving occupational health. The program should be 
continued and expanded as resources become available. 



RELEVANCE OF THE HEALTH HAZARD EVALUATION PROGRAM 

Inputs 

The strategic plan and objectives of the HHE Program are found by the com- 
mittee to be highly relevant, but the committee believes that program performance 
measures are of necessity restricted by available resources. The committee rec- 
ommends that performance measures be reviewed regularly with respect to avail- 
able resources to determine whether more ambitious goals are possible. 

For the most part, the HHE Program uses its resources judiciously to meet its 
mission in the face of a changing economy, the changing nature of HHE requests, 
and increased responsibilities related to emergency response. The committee rec- 
ommends that the HHE Program continue to provide guidance during public 
health emergencies, but also recommends the development of a mechanism to en- 
sure that regular program functions continue during the deployment of key staff. 

The HHE Program could make better use of available surveillance data to as- 
sist in targeting field investigations to recognize previously unknown hazards. 
There is no national occupational health surveillance system, but the HHE Pro- 
gram might influence the development of such a system, perhaps within another 
NIOSH program, such as the Surveillance Branch of the Division of Respiratory 
Disease Studies. 

The committee found the extent and effectiveness of relationships between the 
HHE Program and federal and state health agencies to be variable and recom- 
mends the program work toward consistently effective relationships with these 
different groups. 



Activities 

One of the most important activities of the HHE Program is its response to re- 
quests for evaluation of potential occupational health hazards. The program has 
been a passive recipient of such requests. The committee recommends a more ac- 
tive approach to stimulating valid requests, especially from small businesses and 
underserved populations. Innovative techniques are necessary to acquaint such 
populations with the services offered by the HHE Program. 

Another important activity is priority setting among requests for investiga- 
tions. For that purpose, the HHE Program has developed a triage process to de- 
termine whether a request meets regulatory requirements and whether a site visit 
and full investigation are warranted. The committee finds the development of this 
process a program improvement, but notes that the process is neither well docu- 
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mented nor transparent to the requestor. The eommittee recommends the devel- 
opment of an explicit, transparent written process for triage of requests. 



Outputs 

The major outputs of the HHE Program are reports written in response to re- 
quests. The committee finds that the reports are generally well written, present 
relevant information supported by appropriate documentation, and reflect a high 
level of expertise. However, the committee recommends the development of a 
well-defined quality-assurance program that incorporates expert review from 
elsewhere in NIOSH and externally from the professional and research communi- 
ties. Such a program could ensure consistently high quality outputs. 

An outcome of the HHE Program should be the detection of emerging hazards 
based on requests received. The committee finds no systematic approach to 
achieving such an output and recommends the HHE Program initiate a formal pe- 
riodic assessment of new and emerging hazards. 



IMPACT OF THE HEALTH HAZARD EVALUATION PROGRAM 

Reducing Worker Risk and Preventing Occupational Illness 

In the field of occupational health, evidence suggests that exposure to certain 
chemical or physical agents causes illness, injury, or death. Elimination of expo- 
sure can reduce the number of cases of disease in those already exposed, prevent 
disease in new hires, or both. On the basis of the cause-effect relafionship, at- 
tempts are made to reduce disease risk by reducing or eliminating exposure to 
specific agents. Numbers of cases of disease or numbers of deaths avoided can be 
estimated on the basis of reduction in exposure, but ideally one would have quan- 
titative evidence of the reduction in ilhicss or death at specific investigated work- 
sites or similar workplaces throughout the country. Such quantitative evidence of 
impact is generally unavailable. 



Transferring Program-Generated Information to Relevant Employers and 
Employees Beyond Investigated Workplaces 

The HHE Program uses a wide variety of mechanisms to disseminate informa- 
tion, including informafion posted on the NIOSH Web site; free CDs made avail- 
able to industries, workers, professors, and students; published articles in techni- 
cal, trade, and scientific journals; and, when required. Health Hazard Alerts to 
other government agencies at both state and federal levels, such as the Occupa- 
tional Safety and Health Administration (OSHA). The committee finds, however, 
that the penetrafion of this information into some communities is variable, with 
less penetration into small businesses unaffiliated with trade organizations and 
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underserved populations, including migrant workers. The committee recommends 
the development and use of innovative techniques to reach such populations. 



Influence of (he Health Hazard Evaluation Program on 
NIOSH Research and Policy Development 

The HHE Program docs not have the authority to promulgate regulations, but 
it does inform and support NlOSH-recommcnded guidelines and policies and 
NIOSH testimony regarding proposed OSHA rules. These activities have included 
guidance on a variety of occupational hazards, including those associated with 
tuberculosis, ergonomics, biosolids, latex, indoor air, metalworking fluids, histo- 
plasmosis, hcxavalcnt chromium, body art, hearing loss, and respirator-selection 
decision logic. Thus, the HHE Program has had an impact on a broad array of is- 
sues related to development of NIOSH policy to reduce worker risk associated 
with occupational hazards. As stated earlier, quantitative data are not available to 
determine exact reductions in illness and death (the end outcomes), but there can 
be no doubt that the HHE Program is a major contributor to the development of 
policies expected to lead to a safer working environment {an intermediate out- 
come). 

The HHE Program has also had both direct and indirect impact on NIOSH re- 
search programs as well as the body of scientific knowledge in general. HHE Pro- 
gram reports influence the direction taken by the NIOSH Respiratory Diseases 
Research Program through the identification of unexpected workplace hazards. 
Examples of research influenced by the HHE Program are studies of the respira- 
tory problems caused by flavorings, flock,' waterproofing spray, vaporized vi- 
ruses, and, more recently, nanoparticles. Extensive publication of those studies 
can be found in the open literature. HHE Program staff indicated to the committee 
that their reports also influence research in other NIOSH programs, such as the 
Cancer, Reproductive, and Cardiovascular Diseases Program; the Engineering 
Controls Program; the Exposure Assessment Program; the Hearing Loss Preven- 
tion Program; the Musculoskeletal Disorders Program; and the Personal Protec- 
tive Technology Program. 

Important additional impacts of the HHE Program result from the develop- 
ment of occupational expertise through its training programs. These are major 
contributions, and program impact could be expanded by increasing recruitment 
of trainees and by maintaining contact with training program alumni- Training 
program participants and alumni could be enlisted to provide expertise, assistance 
in identifying emerging hazards, and to assist in maintaining routine program op- 
erations during emergency response. 



' Fine, small-diameter synthetic fibers, breathing of which can cause respiiatory problems. 
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Transfer of Program-Generated Hazard and Prevention Information to the 
Occupational Health Community 

Program-generated hazard and prevention information is transferred in part by 
HHE Program reports and teehnical assistance letters generated in response to re- 
quests for assistance. In addition, HHE Program staff publish in the peer-reviewed 
literature, present research at major academic conferences related to occupational 
health, publish in trade journals, and make presentations at trade-association 
meetings. HHE Program staff have developed compendia of findings and recom- 
mendations about well-understood occupational health risks. Completed compen- 
dia include those on isocyanates, noise, tuberculosis, and lead (NIOSH 2004e, 
1998c, 2001b, and 2001a, respectively). The committee encourages the program 
to continue the development of such compendia as a valuable method of transfer- 
ring information to the general occupational health community. The program also 
participates in a variety of interagency activities related to occupational health. 

The HHE Program is a recognized federal leader in responding quickly and 
effectively to new and emerging hazards, particularly in emergencies. A challenge 
for the program is continuing routine operations while responding to emergencies. 

Despite the activities described above, the committee believes more could be 
done to reach a broader variety of people, trades, and businesses, such as small, 
high-risk residential construction companies, small businesses, and immigrant or 
temporary workers. The committee heard from some state and local health offi- 
cials who were unaware of the existence of the HHE Program. The program 
should not be a secret to those interested in the safety of the workplace. 

Evaluation of the impact of the HHE Program would be greatly enhanced by 
the establishment of a disciplined, after-the-fact evaluation of the relevance, im- 
pact, and quality of responses to HHE investigations. A "foUowback" program 
was initiated on the advice of an earlier evaluation committee, but it should be 
expanded to include more complete information on the impact of the program. 



FUTURE DIRECTIONS OF THE HEALTH HAZARD EVALUATION 

PROGRAM 

The committee's "vision" for the future of the HHE Program is that the pro- 
gram would serve to identify heretofore unrecognized workplace hazards, as well 
as known hazards for which permissible exposure limits or other control measures 
appear inadequate. The HHE Program of the future would continue to do what 
NIOSH can do uniquely— for example, emphasizing health -effects identification 
that combines medical investigation and industrial hygiene (exposure assessment) 
techniques with epidemiological and toxicological perspectives, which may lead 
to the development of new measurement and control techniques. 

To achieve such goals, the HHE Program must become better known in the 
workplace and in the occupational health community. The program can only re- 
spond to requests from others, but such requests cannot be generated if potential 
requestors have not heard of the program. A major challenge will be to make its 
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services more widely known to small businesses, underserved populations, and 
others. 

In the eommittee's vision, the HHE Program of the future would expand its 
role as a major souree of training of occupational health professionals, not only 
for NIOSH, but also for other government agencies, academic institutions, and 
international groups. The program would maintain good working relationships 
with local, state, and national organizations with tics to occupational health so that 
the program would be a primary resource when health problems arise in the 
workplace. The HHE Program of the future would make full use of its recently- 
instituted foUowback program and maintain records of the effectiveness of rec- 
ommended workplace interventions. In that way, HHE Program staff could evalu- 
ate how well the program meets its strategic goals and learn which techniques are 
most effective. Finally, the HHE Program of the future would continue to provide 
guidance and recommendations during public health emergencies. 

In contrast, the HHE Program of the future would not expend its limited re- 
sources doing what others can do, such as conducting routine industrial hygiene 
measurements or providing standardized responses to common problems. The ex- 
pertise in the HHE Program is better used in developing generalized protocols for 
responding to common problems, rather than in investigating routine industrial 
hygiene matters. HHE Program investigations must also be distinguished from 
OSHA compliance inspections and small business consultations. The HHE Pro- 
gram is designed to address emerging hazards, not to supplement the OSHA en- 
forcement process or small-business consultation program. 

The committee's recommendations are summarized in Box S-1. 



SCORING 

The scoring system developed by the Committee to Review NIOSH Research 
Programs is based on a five-point scale in which 5 is the highest score (see Box S- 
2). The committee believes the HHE Program scores somewhere above a 4 for 
both impact and relevance, but notes where the program could be improved. Be- 
cause the committee was required to provide integer scores, the HHE Program 
received a 4 for both relevance and impact. 
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BOX S-1 

Committee Recommendations 

The committee organizes its recommendations in eight categories, presented in an 
order consistent with the HHE Program logic model. The committee does not set pri- 
orities among its recommendations; It prefers encouraging implementation of any of 
them as resources are available. The committee recognizes that implementation of 
many of these recommendations will depend on the availability of resources, but, at 
the request of NIOSH, it refrains from nnaking recommendations regarding resource 
allocations. Therefore, general recommendations are made for program improve- 
ment. The next step for the HHE Program is to consider how resources could be allo- 
cated to allow implementation. 

Recommendation 1: Conduct regular assessments of performance measures to deter- 
mine whether available resources allow more ambitious goals. 

Recommendation 2: Improve the mechanisms by which HHE Program requests are 
sought and prioritized to include a broader array of requests from a wider variety of 
requestors. The program could achieve this through 

a. Systematic use of professional meetings, scientific conferences, scientific litera- 
ture, and surveillance data, including those generated by NIOSH, to assist in pri- 
oritizing field investigations and in recognizing emerging Issues. 

b. Implementing, as part of the triage process, a formal technical assistance 
mechanism to help requestors to formulate valid HHE requests. In cases where 
an HHE is not appropriate or where resource limitations prohibit an investigation, 
technical assistance should Include referral to more appropriate NIOSH divisions 
or government agencies. 

c. Development of an explicit, written process for classifying and prioritizing HHE 
requests. 

d. Better formalizing of the triage process, including the identification of needed ex- 
pertise, and improving the transparency of the process to HHE requestors, while 
maintaining flexibility and speed. 

e. Establishment of formal relationships with organizations representing under- 
served populations, small businesses, and their employees. 

f. Enhancing HHE Program outreach to OSHA national and regional offices and to 
state health and labor departments to better communicate the functions and ac- 
tivities of the HHE Program, increase cooperation with these agencies, and pro- 
vide more complete and timely feedback. 

Recommendation 3: Ensure that recommendations in HHEs are relevant, feasible, effec- 
tive, and clearly explained. Such steps may include 

a. Explanation of the relevance, feasibility, and impact of each recommendation in 
the text of HHE reports. 

b. Priority-setting among recommendations in all reports to indicate those requiring 
immediate action in the targeted workplace. 

c. Debriefing in NIOSH after site visits and report dissemination for determination of 
relevance and impact on a systematic basis (potentially missed opportunities to 
identify emerging health hazards could also be identified). 

d. Modification of the followback surveys for use in assessing the relevance, feasi- 
bility, and impact of recommendations. 

e. Enhancement of internal quality assurance by development of a formal program. 
Consider extemal review of a sampling of recent reports and technical assistance 

letters for scientific content, report completeness, and appropriateness of recommendations. 
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Recommendation 4: Use the HHE Program to develop occupational health professional 
resources. This could be accomplished through 

a. Increased recruitment of new investigators from universities, the Epidemic Intelli- 
gence Service (EIS), the Commissioned Officer Student Training Extern Program 
(COSTEP), occupational medicine residencies, Education and Research Centers 
for Occupational Safety and Health (ERCs), and state and local health depart- 
ments into HHE Program training rotations. 

b. Tracking and mobilizing the extensive talent and commitment represented in the 
HHE Program-trained occupational health workforce. A network of HHE Program 
alumni could be fostered to help to develop HHE opportunities, A program-level 
advisory board could assist the program in leveraging resources, serve a recruit- 
ing and retention function, assist in identifying emerging issues, and provide ex- 
pert advice. 

c. Engagement and use of ERCs and other university-based training programs to 
involve trainees in HHE field investigations. 

d. More formal collaboration with ERC faculty and other extramural researchers to 
assist in field investigation, dissemination, and training opportunities. 

Recommendation 5: Develop a proactive, comprehensive information-transfer strat- 
egy for HHE Program outputs with better approaches to reaching wider audiences, 
including traditionally underserved populations. The HHE Program could 

a. Use innovative techniques to reach small businesses and underserved popula- 
tions, creating a broad array of mechanisms for communicating with diverse con- 
stituencies and attending to issues of literacy, language, and national-origin bar- 
riers. The effectiveness of applied outreach should be evaluated in a formal 
manner. 

b. Improve the searchabilily of the online HHE search engine by developing a list of 
standardized key words (an alphabetized list of hazards and diseases would be 
beneficial). 

c. Develop distribution mechanisms that are not Internet-dependent to complement 
Internet distributions. 

d. Disseminate HHE results more broadly to groups likely to be affected, including 
distribution of HHE reports in the geographic regions where investigations are 
conducted. 

e. Increase efforts to compile compendia of findings (such as those developed for 
isocyanates, noise, tuberculosis, and lead) when generalized process-oriented 
findings can be gleaned from the experience of the HHE Program in a variety of 
settings, 

f. Develop Improved methods of outreach to stakeholders so that workers and 
workplaces affected by new and emerging occupational health problems will be 
alerted quickly. 

g. Supplement program outreach efforts by using community and small-business 
groups to translate HHE results and findings for their constituencies. 

h. Leverage existing NIOSH. Centers for Disease Control and Prevention (CDC), 
and Department of Health and Human Services (HHS) resources to enhance 
technology transfer. 

i. Evaluate, in a formal manner, the effectiveness of information -transfer programs, 
including knowledge transfer to employers and employees not investigated. 

Recommendation 6: Develop more extensive formal linkages and mechanisms with 
other parts of NIOSH, CDC, and HHS to enhance the capacity for involvement In pol- 
icy-relevant impacts through 

a. Promotion and increase In direct communication, especially with OSHA and stale 
occupational s afety and health agencies. 
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b. Alerts to NIOSH and CDC about HHEs that are relevant to policy-making outside 
the CDC system. 

c. Continued regular use of the National Occupational Research Agenda (NORA) 
sector councils and the NIOSH Board of Scientific Counselors to disseminate in- 
formation about the HHE Program. 

d. Pursuit of a change in the HHE Program's legislative and regulatory authority to 
improve the capacity to identify hazards in need of HHEs, improve the ability to 
gain entrance to facilities when requested by treating physicians or community 
representatives, and address exposures other than chemical agents. 

Recommendation 7: Initiate formal periodic assessment of new and emerging haz- 
ards. To accomplish this, the HHE Program could 

a. Evolve from a program that passively receives requests to a proactive program 
that seeks opportunities for field investigations. 

b. Develop systematic approaches to identify hazards where OSHA permissible ex- 
posure limits are inadequate or nonexistent, to identify unknown hazards, and to 
identify known hazards encountered under new circumstances. 

c. Establish and periodically review a tickler file of inconclusive or unexpected 
evaluation results to determine whether new trends or problems may be emerg- 
ing. 

d. Periodically meet with intramural and extramural research scientists and stake- 
holders in government, academe, labor, and industry to discuss specific unre- 
solved evaluations, to review aggregate findings, and to solicit input about new or 
emerging hazards or interventions. 

Recommendation 8: Continue to provide guidance and recommendations during pub- 
lic health emergencies. To accomplish this, the HHE Program could 

a. Work with NIOSH management to remain diligent to avoid negative impact on 
routine activities of the HHE Program as a result of emergency response activi- 
ties. 

b. Develop a mechanism, such as the enlistment of help from training program par- 
ticipants and alumni, to ensure continuation of routine operations in the absence 
of staff involved in emergency response. 
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BOX S-2 

Committee Scoring Criteria 

Relevance 

5 = Activities are in high-priority subject areas and NIOSH is significantly engaged in ap- 
propriate transfer activities for completed projects/reported results. 

4 = Activities are in priority subject areas and NIOSH is engaged in appropriate transfer 

activities for completed projects/reported results. 

3 = Activities are in high-priority or priority subject areas, but NIOSH is not engaged In 

appropriate transfer activities; or activities focus on lesser priorities but NIOSH is en- 
gaged In appropriate transfer activities. 

2 = Activities are focused on lesser priorities and NIOSH is not engaged in or planning 

some appropriate transfer activities. 

1 = Activities are not focused on priorities and NIOSH is not engaged in transfer activi- 

ties. 

Impact 

5 = Activities have made major contribulion(s) to w^orker health and safety on the basis of 

end outcomes or w^ell-accepted intermediate outcomes. 

4 = Activities have made some contributions to end outcomes or well-accepted Interme- 

diate outcomes. 

3 = Activities are ongoing and outputs are produced that are likely to result in improve- 

ments In worker health and safety (with explanation of why not rated higher). Well- 
accepted outcomes have not been recorded. 

2 - Activities are ongoing and outputs are produced that may result in new knowledge or 

technology, but only limited application Is expected. Well accepted outcomes have not 

been recorded. 
1 = Activities and outputs do not result in or are NOT likely to have any application. 
NA - Impact cannot be assessed; program not mature enough. 
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CHARGE TO THE COMMITTEE 

In 2005, the National Institute for Occupational Safety and Health (NIOSH) 
requested the National Academics review a series of up to 15 of its research pro- 
grams. The Committee to Review the NIOSH Health Hazard Evaluation (HHE) 
Program was formed in 2007 to carry out an independent evaluation of the HHE 
Program under the following charge: 

The National Academies will appoint an ad hoc committee to carry out an 
independent evaluation of the impact, relevance, and future directions of 
the NIOSH Health Hazards Evaluation (HHE) Program. The committee 
will evaluate not only what the NIOSH HHE program is producing, but 
will also determine whether it is appropriate lo credit NIOSH activities 
with changes in workplace practices, hazardous exposures, and/or occupa- 
tional illnesses and injuries, or whether the changes are the result of other 
factors unrelated to NIOSH. 

In conducting its assessment, the committee will evaluate 

I . The impact of the program on 

• reducing worker risk and preventing occupational illness in inves- 
tigated workplaces; 

• transferring program-generated information to relevant employers 
and employees beyond the investigated workplaces; 

• NIOSH research and policy-development programs; and 

• the activities of regulatory agencies, occupational safety and health 
professionals and organizations, state and local health agencies, 
and others in the occupationiil liealth community, as achieved by 
transferring program-generated hazard and prevention information. 
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2. The relevance of the program in addressing current and emerging 
workplace health hazards. 

The evaluation committee will provide quantitative assessments in the 
form of integer scores (on a scale of 1-5) for both the relevance and impact 
of the program in addressing workplace health hazards, to be accompanied 
by qualitative assessments of all the categories above. The committee will 
develop its own methodology for the evaluation, guided by the methodol- 
ogy and framework developed by the Committee to Review NIOSH Re- 
search Programs where appropriate. 

The HHE Program does not conduct traditional research as do other NIOSH 
programs. It is mandated to respond to requests for assistance to identify specific 
workplace conditions that pose health hazards to workers (NIOSH, 2007b). The 
charge to this committee was therefore modified from that given for other evalua- 
tions in this series. 



Definitions of Impact and Relevance 

In developing its evaluation methodology, the committee carefully studied the 
charge and determined definitions for impact and relevance. For this evaluation, 
impact is defmed more broadly than for the evaluations of other NIOSH pro- 
grams. This committee is asked to deteraiine whether the HHE Program contrib- 
utes to occupational health in the workplace, as well as whether the program posi- 
tively affects policy, other research programs, and the health and safety commu- 
nity in general. The charge to the committee specifically includes transfer of in- 
formation as part of impact, although the methodology and evaluation framework 
(hereafter called the Framework Document) developed by the Committee to Re- 
view NIOSH Research Programs (hereafter called the Framework Committee) 
tends to include transfer in its definition of relevance. The committee considers 
transfer activities important for both relevance ("Is the program doing the right 
things?") and impact ("Is what the program is doing effective?"). 

The Framework Document defines a relevant program as an integrated pro- 
gram involving interrelated surveillance, research, and transfer activities. The 
evaluation criteria include the severity or frequency of hazards addressed and 
number of people at risk; the extent to which gender-related issues and those of 
vulnerable populations (hereafter described as "underserved populations"; see 
Box 1-1 for definition) are addressed; the extent to which the health and safety 



' Public health surveillance is the "ongoing, syslematic collection, analysis, interpretation, and 
dissemination of data regarding a health-related event for use in public health action to reduce 
morbidity and mortality and to improve health. Data disseminated by a public health surveillance 
system can be used for immediate, program planning and evaluation and formulating research 
hypotheses" (MMWR, 200 lb:2). 
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BOX 1-1 

Vulnerable or Underserved Populations 

Vulnerable populations are defined in the Framework Document (Appendix A) as 
groups of workers who have biologic, social, or economic characteristics that place them 
at increased risk for work-related conditions or on whom inadequate data have been col- 
lected. They include disadvantaged minorities, disabled persons, low-wage workers, and 
non-English-speakers for whom language or other barriers present health or safety risks. 
Undocumented workers also fall within this category. This evaluation committee uses the 
term "underserved populations" when referring to vulnera ble populations. 

needs of small businesses arc addressed; the "life stage"" of problems being ad- 
dressed; and the structure and content of the program. 

The criteria for relevance as defined in the Framework Document are as appli- 
cable to the HHE Program as they arc to traditional NIOSH research programs. 
The committee is to determine whether occupational health issues addressed by 
the HHE Program arc the most serious in gravity, or the most frequent among the 
spectrum of issues to which resources n:iight be applied. They also are to deter- 
mine if the HHE Program addressed the needs of underserved populations and 
small businesses. HHEs arc not the same as investigator-initiated research such as 
that sponsored by the National Institutes of Health. HHEs are considered part of 
the federal government's public health surveillance activities, similar to outbreak 
investigations of infectious diseases in communities, and thus are exempt from the 
requirement of review by the Institutional Review Board that other forms of hu- 
man-subjects research are subject to. For this reason, the committee considers the 
extent to which HHEs respond to the needs of the requesters to be an additional 
measure of relevance. 

In all its criteria, the Framework Document directs evaluation committees to 
consider how ^'program research" is relevant or has impact. This committee fmds 
that substituting "program activities" for "program research" is an adequate modi- 
fication in most cases, including within the Framework Committee's scoring cri- 
teria. Box 1-2 provides the scoring criteria used by the committee in the determi- 
nation of rankings for relevance and impact. 



" The "life siage" of an issue is delennined by identifying the type of activities that should be un- 
dertaken at a given time to correcl an issue. For example, as health effects are understood, efforts 
should shift from efficacy to intervention and intervention-effectiveness research. 
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BOX 1-2 

Scoring Criteria 

The following is the scoring criteria used by the committee In its evaluation of rele- 
vance and impact of the HHE Program. The criteria are based on those developed by the 
Framework Committee (see Appendix A), but have been slightly modified to accommo- 
date the more applied nature of the HHE Program. 

Scoring Criteria for Relevance 

5 = Activities are in high-priority subject areas and NIOSH is significantly engaged in ap- 
propriate transfer activities for completed projects/ re ported results. 

4 = Activities are in priority subject areas and NIOSH is engaged in appropriate transfer 

activities for completed projects/reported results. 

3 = Activities are in high-priority or priority subject areas, but NIOSH is not engaged in 

appropriate transfer activities; or activities focus on lesser priorities but NIOSH is en- 
gaged in appropriate transfer activities. 

2 = Activities are focused on lesser priorities and NIOSH is not engaged in or planning 

some appropriate transfer activities. 

1 = Activities are not focused on priorities and NIOSH is not engaged in transfer activi- 

ties. 

Scoring Criteria for Impact 

5 = Activities have made major contribution(s) to worker health and safety on the basis of 

end outcomes or well-accepted intermediate outcomes. 

4 = Activities have made some contributions to end outcomes or well-accepted interme- 

diate outcomes. 

3 = Activities are ongoing and outputs are produced that are likely to result in improve- 

ments in worker health and safety (with explanation of why not rated higher). Well- 
accepted outcomes have not been recorded. 

2 = Activities are ongoing and outputs are produced that may result in new knowledge or 

technology, but only limited application is expected. Well accepted outcomes have not 

been recorded. 
1 = Activities and outputs do not result in or are NOT likely to have any application. 
NA = Impact cannot be assessed; program not mature enough. 



THE EVALUATION PROCESS 

Information Gathering 

The committee reviewed material submitted by the HHE Program in the form 
of an "evidence package" that describes the HHE Program's legislative founda- 
tions, organizational structure and management, resources, activities, and outputs 
(NIOSH, 2007b). Contributions to improvements in worker health arc summa- 
rized in the form of narratives related to specific workplace issues. Results of pre- 
vious evaluations, and lists and examples of HHE Program reports are provided. 
HHE Program staff also responded during meetings and in writing to numerous 
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questions from the committee and participated in a telephone conference with 
committee members to discuss HHE Program emergency response activities. Ap- 
pendix B is a list of materials provided by the HHE Program to assist the evalua- 
tion process. 

To deliberate its charge, the committee met four times. Two meetings were 
held in Washington, DC (September and December 2007), and two in Irvine, 
California (January and February 2008). The first three meetings included brief- 
ings from NIOSH and HHE Program staff and stakeholders. Meeting agendas, 
including names and affiliations of those who provided input, are provided in Ap- 
pendix C. Committee members discussed pertinent issues with several stake- 
holders on two panels representing the workplace and public sectors during the 
December 2007 meeting. In January 2008, the committee heard from stakeholders 
regarding four specific HHEs (NIOSH, 2004a, 2005b, 2005f, 2007j). Meetings 
were held on both East and West Coasts to facilitate stakeholder participation and 
attendance. Subsets of the committee held numerous phone conferences and a 
meeting in Cineinnafi, Ohio, to prepare the final report. 

The committee found a general paucity of quantitative data showing the im- 
pact of HHE Program activities. For this reason, the committee made great efforts 
to hear from those who could reasonably be expected to benefit from HHE Pro- 
gram activities. The committee invited individuals representing small and large 
businesses; researchers in academe; local, state, and federal agencies; labor un- 
ions; and workers' rights organizations and advocacy groups to participate in dis- 
cussions regarding the relevance and impact of HHE Program activities. All were 
asked to speak of their positive and negative experiences with the HHE Program 
and the uscfijlness and effectiveness of any resulting HHE recommendations. 
HHE Program stakeholders unable to travel to meetings were invited to partici- 
pate in discussions via teleconference. The committee found it necessary to rely 
on this anecdotal evidence when no other evidence was available, but was careful 
to provide citations in the report when doing so. 

To receive a broader range of stakeholder input, the committee also requested 
public input via an online questionnaire (sec Appendix C). The questionnaire was 
announced via email to over 500 stakeholders. Responses could be submitted 
online, by e-mail, or standard mail, and the option to respond anonymously was 
available. Questionnaire announcements were distributed to individuals suggested 
by the committee, as well as to those on lists provided by the HHE Program and 
generated by other evaluation committees of this evaluation series. The HHE Pro- 
gram provided links to the questionnaire on its Internet site, and the request for 
input was also announced through a variety of means including a NIOSH listscrv 
for state health and labor departments that partner with NIOSH on occupational 
health surveillance; a listscrv for occupational health practitioners run by the Uni- 
versity of North Carolina; the American Industrial Hygiene Association weekly e- 
newsletter: and NIOSH cNews, a monthly newsletter (NIOSH eNews, 2008). The 
committee received 57 responses. A summary of stakeholder rccommendafions 
and identified emerging issues is provided in Appendix D (derived from Stake- 
holder Response Table, 2008). The committee found stakeholder insights thought- 
ful and invaluable during deliberations. 
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Additional stakeholder input related to HHE Program emergency response 
activities was solicited (see Appendix C). Two responses were received as a result 
of those efforts. 



Period Evaluated 

Because the HHE Program has evolved since its inception, and because there 
have been multiple evaluations of this program in the past (see Chapter 2), the 
committee focuses on current program processes and activities. The committee 
chooses to emphasize the period between 1997 and 2007 in its evaluation, how- 
ever many program activities before and after this timeframe are considered. 



REPORT ORGANIZATION 

This report is organized into five sections. This chapter orients the reader by 
outlining the committee review process. Chapter 2 provides a short history of the 
HHE Program and its legislative authority, and provides a description of the pro- 
gram's inputs, activities, and outputs. The committee's evaluation of the HHE 
Program is found in Chapters 3 and 4, which are organized in a manner consistent 
with the program's logic model (presented in Chapter 2). In Chapter 3, the com- 
mittee describes the relevance of the program's strategic plan, use of resources, 
program activities, outputs, and emergency response activities. The committee's 
findings are organized and described under each of these categories. In Chapter 4, 
the committee evaluates the impact of the program. The chapter includes four sec- 
tions corresponding to the four parts of the committee's charge with respect to 
impact: reduction in risk and prevention of occupational illness in investigated 
workplaces; transfer of information beyond investigated worksites; influence on 
NIOSH research and policy; and impact on others in the occupational health 
community. A fifth section discusses the ipact of program emergency response 
acfivitics. Finally, in Chapter 5, the committee describes its vision of what the 
HHE Program of the future would be, and summarizes recommendations made in 
Chapters 3 and 4. 
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INTRODUCTION 

The primary charge to the Health Hazard Evaluation (HHE) Program, as de- 
scribed in Section 20(a)(6) of the 1970 Occupational Safety and Health Act (OSH 
Act; P.L. 91-596),' is to respond to written requests to investigate workplace 
health hazards. The program conducts field evaluations and consultations, re- 
sponds to emergencies, and provides training for health professionals. These ac- 
tivities are conducted by a staff trained as gencralisls in occupational medicine, 
epidemiology, industrial hygiene, and by some specialists in areas such as pulmo- 
nary medicine (sec Box 2-1 for definitions of some of these terms). HHE Program 
stakeholders include those at worksites where HHEs are conducted, workers and 
employers at similar workplaces, regulatory bodies, occupational and public 
health practitioners, other National Institute for Occupational Safety and Health 
(NIOSH) programs, and the research community more broadly. 

The Committee to Review the HHE Program opted to separate the compo- 
nents to be evaluated as described in the HHE Program logic model (Figure 2- 
1). To be evaluated are the program's inputs (strategic goals and objectives, pro- 
gram resources and communication from stakeholders), activities (responses to 
HHE requests), outputs (such as HHE reports and NIOSH published documents), 
intermediate outcomes (such as reductions in hazardous exposures), and end out- 
comes (reduction in occupational illness). A simplified version of the logic model 
(shown in Appendix A, Figure 2) has been used in reviews of the other NIOSH 
research programs. 

This chapter provides a historical context for the program and describes HHE 
Program inputs, activities and outputs. The relevance of these components is as- 
sessed in Chapter 3. Stakeholder response to and impacts resulting from HHE ac- 
tivities are discussed primarily in Chapter 4. 



' P.L. 91-596, 91st Cong., S.2193, December 29, 1970, as amended ihroiigh January 1, 2004 
(http://www, osha.gov/pls/oshaweb/owadisp. show_document?p_tablc=OSHACT&p_id=2743, 
accessed March 21, 2008). 
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BOX 2-1 

Definitions of Common Terms 

Epidemiology The study of the distribution of, and the physical, biological, social, cul- 
tural, and behavioral factors that influence health-related states or events in specified 
populations. Epidemiology also includes the application of ttiis study to control of tiealtti 
problems (SOURCE: Last, 2001). 

Industrial hygiene The science and art devoted to the recognition, evaluation, and con- 
trol of the environmental factors or stresses arising from or in the workplace that may 
cause sickness, impaired health and well-being, or significant discomfort and inefficiency 
among workers or among persons in the community; the profession that anticipates and 
controls unhealthy conditions of work to prevent illness among employees (SOURCE: 
Last, 2001). 

Occupational Medicine The specialized practice of medicine, public health, and ancil- 
lary health professions in an occupational setting in order to promote health and prevent 
occupalionally related disease and injury (SOURCE: Last, 2001). 

Surveillance The systematic collection, analysis, interpretation, and dissemination of 
data regarding a health-related event for use in public health action to reduce morbidity 
and mortality and to improve health (SOURCE: MMWR, 2001b:2). 



HISTORICAL OVERVIEW 

The Occupational Safety and Health Administration (OSHA) is the federal 
agency charged with protecting worker safety and health by setting and enforcing 
workplace standards. However, many hazards, such as noise, certain chemicals 
used in commerce, or conditions that cause musculoskeletal disorders are not 
regulated by standards, and many existing standards may be obsolete. When oc- 
cupational illness is suspected, OSHA workers and employers rely on the HHE 
Program to identify illness-causing hazards and to recommend control solutions. 

During the June 19, 1968, Senate hearings on the proposed OSH Aet, Phillip 
R. Lee, MD, Assistant Secretary for Health and Scientific Affairs, Department of 
Health and Human Services (HI IS), testified: 

We are dealing with hidden health hazards. Many occupational illnesses occur 
only after long periods of exposure to one or more hazards in the environment 
so that the link between hazard and frank disease is not dramatic and overt. . . 
. Moreover, a large part of the problem lies in small establishments which em- 
ploy fewer than 100 workers. . . . The vast majority of these workplaces have 
no safety engineers, doctors, nurses, hygicnists, or meaningful safety and 
health programs. . . . The villain is not greed; it is ignorance. One of the basic 
objectives of the legislation before you today. . . .is to remove those barriers of 
ignorance that result in so much needless suffering and economic loss (U.S. 
Congress, Senate, Committee on Labor and Public Welfare, 1968:270-271). 



Prepublication Version - Subject to Further Editorial Revision 







Mission: 

To protect worker huHh through problem solving, research, risk communlcillon, 

and dissemination of flrxllngs and recommendations by reBpondlng to extamal 

requests forliaxardsveluattons and technical aBGietence 



lla 





IstHHi! 



I 



Hlim 



m 



^ m 

i 




I 



o 

l-H 

Ah 

w 
X 
X 



o 



T3 
O . 



o — 

-1 X 

T O 



3 



Urn in 






■3 



^ 



to 



St 



22 THE HEALTH H.IZARD EVALUATION PROGRAM AT NIOSH 

Congress ultimately recognized that the nation needed a mechanism to inves- 
tigate health hazards in the workplace, regardless of compliance with specific 
standards. The OSH Act of 1970 (P.L. 91-596) included provisions in Section 20 
(Research and Related Activities, 29 USC 669) to be carried out by the Secretary 
of Health, Education and Welfare through a new NIOSH, which was established 
in Section 22 of the act."' 

Section 20(a)(6) provides that 

The Secretary of Health and Human Services shall publish within six 
months of enactment of this Act and thereafter as needed but at least an- 
nually a list of all known toxic substances by generic family or other use- 
ful grouping, and the concentrations at which such toxicity is known to 
occur. He shall determine /f)//(:)vt'//;g a written request by any employer or 
authorized representative of employees, specifying with reasonable par- 
ticularity the grounds on which the request is made, whether any sub- 
stance normally found in the place of employment has potentially toxic ef- 
fects in such concentrations as used or found [emphasis added]; and shall 
submit such determination both to employers and affected employees as 
soon as possible. If the Secretary of Health and Human Services deter- 
mines that any substance is potentially toxic at the concentrations in which 
it is used or found in a place of employment, and such substance is not 
covered by an occupational safety or health standard promulgated under 
section 6, the Secretary of Health and Human Services shall immediately 
submit such determination to the Secretary [of Labor], together with all 
pertinent criteria. 

The organization to conduct these investigations became the HHE Program, 
which responds to requests from employers, employees, and their representatives, 
and other agencies to conduct workplace evaluations. Through HHEs, NIOSH 
identifies current health hazards and makes recommendations to reduce exposures 
and prevent disease and disability. HHEs may be conducted in any private, 
federal, or other governmental workplace. Authority is governed by laws (P.L. 
91-596,-' P.L. 95-164^), federal regulations (29 CFR 1960,^ 42 CFR 85^), and 



" hitp://www. osha.gov/pls/oshaweb/owadisp. show_document?p_table=OSHACT&p_id=3374 

(accessed March 21, 2008). 

' 20 USC (.7 1 

(http://www.osha.gov/pls/oshaweb/owadisp. show_documeni?p_table=OSHACT&p_id^3376, 

accessed March 21, 2008). 

* 20 CFR I960, Basic Program Elements for Federal Employees Occupational Safety and Health 

Programs and Related Matters. Section 1960.35 of these regulations describes the procedures for 

requesting lIMEs in federal agency workplaces. NIOSH follows the procedures outlined in the 

regulations governing HHEs (42 CFR 85) when evaluating Federal ageney workplaces 

(hitp://www.' .osha.gov/pls/oshaweb/owadisp. show_document?p_table^STANDARDS&p_id= 1 1 2H 

4, accessed October 1 0, 2008). 

^ 29 USC 667 [Section 18(c)(6)], 29 USC 668 [Section 191, and 29 USC 609 [Section 20(a)(6)l. 

"^Section 501(a). 

^Section 1960.35. 
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Hxecutive Order 12196 of February 26, 1980. The language of the law is 
important because it implies that the IlflE Program should address hazards not 
heretofore recognized, or levels of exposure not previously deemed hazardous. 
The present committee concludes that Congress intended to fill data gaps present 
in 1970, when OSHA adopted wholesale as permissible exposure limits (PELs) 
(Robinson, 1991) many of the existing American Conference of Governmental 
Industrial Hygienists (ACGIH) threshold limit values (TLVs) {ACGIH, 1989). 
NIOSH was given authority to enter the workplace following a written request, 
and legislation required certain employers to cooperate with an investigation by 
allowing exposure measurements, making employees available for medical 
evaluation, and providing available information. While right of access resembles 
that for an OSHA inspection, the HHE Program conducts evaluations rather than 
inspections. There are important differences. OSHA inspects for known hazards 
and seeks to enforce safety and health regulation by issuing citations and fines as 
appropriate. By contrast, the HHE Program has not been given regulatory 
authority; its mission is intended to evaluate unknown situations with the goal of 
identifying previously unrecognized hazards and developing new control 
strategies. 

Who may submit requests for investigations is defined by law: the request 
must be from an employer; a union; an employee representing at least two other 
employees; a single employee if the work area of concern has three or fewer em- 
ployees; a federal agency health and safety committee, or federal employees not 
covered by such a committee; or the secretary of labor. The legislative authority 
and limitations are important input to the HHE Program and arc also external fac- 
tors that affect the ability of the program to maximize its impact. Although au- 
thority has been broadly interpreted since enactment. NIOSH is legislatively con- 
ferred authority for investigation of only a specific subset of occupational health 
and safety conditions. NIOSH has the ability to compel workplace entry only for 
toxic substance exposure. When entry is not an issue, the HHE Program evaluates 
the full spectrum of hazards. The committee infers that the original vision in- 
cluded the notion that chemical exposures that did not violate existing OSHA 
PELs might be causing health problems that could be identified through medical 
and exposure evaluation. NIOSH was expected both to intervene in the requesting 
workplace and to provide data to support new or modified PELs or other health 
standards. 

Over nearly four decades since the act was passed, the scope of hazards that 
the HHE Program addresses has expanded beyond chemical exposures. The scope 
of activities to which HHE resources are applied was also extended to provide 
technical assistance to requesters from other governmental agencies such as 
OSHA, the U.S. Postal Service, and the U.S. Department of Transportation. 



^Requests fori 11 (Fs. 
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INPUTS 

As noted in the HHE Program logic model (Figure 2-1), inputs include plan- 
ning and the program's use of resources. A challenge facing the HHE Program is 
balancing the need to be responsive to individual requests against the need to de- 
velop strategic approaches to identifying emerging health threats in the work- 
place. The HHE Program conducted a strategic planning process in the mid-1990s 
that "focused largely on internal process issues" {NIOSH, 2007b:29). In 2007, the 
program revised its strategics in response to a 2006 Board of Scientific Counsel- 
ors (BSC) program evaluation and guidance from the NIOSH OtTice of the Direc- 
tor (CD). 

The mission of the HHE Program is "to protect worker health through prob- 
lem solving, research, risk communication, and dissemination of findings and 
recommendations by responding to external requests for hazard evaluations and 
technical assistance" (NIOSH. 2007b: 16). To that end, the program established 
the following strategic goals: 

1. Prevent occupational illnesses through reduced exposure to workplace 
hazards; 

2. Promote occupational safety and health research on emerging issues; and 

3. Protect the health and safety of workers during public health emergencies 
(NIOSH, 2007b:l). 

Intermediate goals and performance measures were established in 2007 in re- 
sponse to the BSC 2006 evaluation and are described and assessed in Chapter 3. 
The HHE Program stated that performance measures will be reviewed annually 
and revised as appropriate (NIOSH, 2007b). 

The strategic goals and activities of the HHE Program arc influenced by a variety 
of factors, including program and agency resources, legislative mandates, HHE re- 
quests, and stakeholder needs. The Framework Document (Appendix A) divides in- 
puts into two categories: production and planning. Production inputs (also called pro- 
gram resources in the Framework Document) include program structure and man- 
agement, funding, staffing, and physical facilities. Planning inputs include input from 
stakeholders, surveillanee and intervention data, and risk assessments. 

What follows are brief descriptions of various inputs to the HHE Program. 
These descriptions are intended to orient the reader to the internal and external 
factors that shape the program. More detail about each of these inputs is provided 
as warranted throughout the report. 



Production Inputs 

Program Organization 

The HHE Program summarized the organization of its activities in the logic 
model shown in Figure 2-1 (assessed in Chapters 3 and 4 of this report). The pro- 
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gram's role in the protection of worker health and safety during public health 
emergencies is not reflected in the logic model, but is included as an important 
element in the program's strategic goals. This is also addressed in Chapters 3 and 4. 

The HHE Program mission is carried out by employees of the Hazard Evalua- 
tions and Technical Assistance Branch (HETAB) in the Division of Surveillance, 
Hazard Evaluations, and Field Studies (DSHEFS) in Cincinnati, Ohio, and the 
Field Studies Branch (FSB) in the Division of Respiratory Disease Studies 
(DRDS) in Morgantown, West Virginia. The administrative lead of the HHE Pro- 
gram resides within HETAB. The HHE Program coordinator also serves as the 
HETAB branch chief. All HHE requests are logged and tracked within HETAB, 
which serves as the primary point of contact within the HHE Program for OSHA 
and for state and local agencies. 

There appears to be considerable support from and collaboration with staff 
throughout NIOSH on specific investigations. The HHE Program described col- 
laboration with NIOSH scientists engaged in many of the priority research areas 
established by the National Occupational Research Agenda (NORA) including 
both cross-sector (health outcome) and sector-specific (industry-related) priorides 
(NIOSH, 2007b). According to the HHE Program, interbraneh coordination 
within the program is facilitated through "joint participation by conference call in 
regular meetings to discuss and assign incoming HHE requests and by informal 
exchanges about procedural, policy, and scientific issues" (NIOSH, 2007b). 

The HHE Program staff informed the committee that the program regularly 
partners with the NIOSH Division of Applied Research and Technology and with 
the Health Effects Laboratory. To fulfill its responsibilities in the area of emer- 
gency response, the HHE Program partners with the Emergency Response and 
Preparedness Branch of the NIOSH OD. In a small number of cases, researchers 
from other NIOSH programs may be given primary responsibility for carrying out 
an HHE. When needed technical expertise is unavailable within NIOSH, the HHE 
Program works with experts in other government agencies or contracts with indi- 
viduals in the private sector. These collaborations are essential in view of the di- 
versity of potential workplace hazards that require evaluation. 



Budget 

Table 2-1 shows the HHE Program budget by branch and class over fiscal 
years 2000 through 2007. The committee did not consider an evaluation of the 
adequacy or appropriateness of budget allocations as part of its charge, nor was it 
given the data to conduct such an evaluation. The committee considers funding 
only in'tcrms of the resources available to the program. Aspects of the program 
budget arc discussed in greater detail as warranted throughout the report text. 

HETAB is the administrative home of the HHE Program and bears the costs 
for program administration. HETAB logs and tracks all HHE requests, maintains 
a central file of HHE requests and reports, and carries out routine communication 
functions, such as notifying OSHA and state and local health departments of HHE 



Prepitblication Version - Subject to Further Editorial Revision 



26 



THE HEALTH HAZARD EVALUATION PROGRAM AT NIOSH 



requests. It also prepares supporting documentation for Office of Management 
and Budget (OMB) approval pursuant to the Paperwork Reduction Act. 

' The NIOSH OD allocates funds to the DSHEFS and DRDS, which then allo- 
cate funds to their respective branches (NIOSH, 2007b). Approximately $7.7 mil- 
lion were allocated to the HHE Program in 2007 (approximately 3 percent of the 
total NIOSH budget). Approximately 77 percent of the funding covers personnel 
costs, while the remainder is intended for discretionary spending. During the past 
10 years, most of the program budget has come from the NIOSH base budget, 
though some funding has been received from earmarks, to offset expenses arising 
from emergency response activities, or was received from NORA research fund- 
ing to support core activities. 



TABLE 2-1 Budget for the HHE Program by Branch and Class: Fiscal Years 2000-2007 



Year 



Class (thousands of dollars) 



Salaries 



Benefits Travel 



Mail 



Mobile 
Communi- 
cations 



Printing | Contracts 



Supplies 



Equip- 
ment 



Fellows 



Hazard Evaluations and Technical Assistance Branch 
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200 1" " 
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2002 
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2003 
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2004 
2005 "" 
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1,124 "^ 


2006 
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2007 
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1,176 



Field Studies Branch 



2000 



935 



232 



2001 
'2002 
2003' 
2004' 
2005" 
2006 
2007" 



1,324 ! 320 

1.402 ' 335" 

_' 3*^0 
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496 

395 

_1 400 

SOURCE: NlOsfl (2007hj." 
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5' 

21 
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Personnel 

As shown in Figure 2-2, there has been a substantial reduction in the total 
number of full-time equivalents (FTEs) since apeak in 2001. Some of this decline 
can be attributed to reorganization by the Centers for Disease Control and Preven- 
tion (CDC) involving clerical staff, but staff reductions within HETAB alone ap- 
proach 20 percent. Periodic staff increases have occurred since 2001, commensu- 
rate with specific projects,' such as those related to flavorings. The number of 
FTEs in 2007 was 61 (NIOSH, 2007b). 
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Staff members are generalists in the areas of occupational medicine, epidemi- 
ology, industrial hygiene, ergonomics, engineering, behavioral science, pulmo- 
nary and other areas of medicine, toxicology, occupational health nursing, and 
statistics. Communication with HHE Program staff indicates that the distribution 
of program staff by discipline/role within HETAB and FSB in the disciplines 
needed to conduct the HHEs, especially given the collaborative efforts within 
NIOSH(NIOSH,2007h). 

Table 2-1 indicates a relatively flat budget in real dollars for the HHE Pro- 
gram over the past eight years. Personnel costs have remained nearly constant 
over this period of time, while contract costs have shown considerable variation. 
Thus, it appears that the relatively flat budget can be associated with decreases in 
FTEs for the program over the same timeframe. 




,^ .^ ^^ / ^'i^ / r/ ^ ^# / 



I HETAB D FSB 



FIGURE 2-2 Total HHE Program FTEs. 
SOURCE: NIOSH (2007b). 



Facilities 



The HHE Program maintains offices and facilities for the storage and mainte- 
nance of scientific equipment used in field investigations at both HETAB and 
FSB facilities. Equipment includes sampling and monitoring equipment for expo- 
sure assessment, personal protective equipment for HHE investigators, and logis- 
tical support equipment and supply. A trailer designed as a staging facility for 
field invcsfigations during emergency response and large-scale nonemergency 
field investigations is maintained at the program's Cincinnati facilities. A medical 
trailer in Morgantown is equipped to assess pulmonary effects of workplace expo- 
sures. 

The HHE Program does not maintain its own suite of chemical and biological 
analytical equipment, but relics on resources elsewhere in NIOSH or on NIOSK- 
funded contract laboratories for sample analyses. 
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Planning Inputs: Previous Evaluations 

Between 1972 and 2006, the HHE Program received planning input in the 
form of 12 internal and external evaluations (NIOSH, 2007b). Evaluation methods 
included expert review panels, key informant interviews, and customer satisfac- 
tion surveys. The three most recent evaluations were conducted by the Research 
Triangle Institute (RTI, 1997) and the NIOSH BSC (1997. 2006). While the 
committee reviewed these evaluations as an aid to understanding the evolution of 
the HHE Program and its elements, it was not part of the committee's charge to 
provide an assessment of these evaluations. Nonetheless, pertinent elements of the 
prior evaluations and the HHE Program responses arc discussed in more detail in 
Chapters 3 and 4. 

The 1997 review, developed through a contract with RTI, elicited feedback 
from a wide range of stakeholders. Problems or classes of criticisms identified 
include 

• lack of timeliness 

• procedural bias (specifically, concerns that employers had greater input 
into the process than employees) 

• overemphasis on routine investigations, and 

• lack of practicality in recommendations (NIOSH, 2007b: Appendix 2.3, 
14). 

RTI recommendations led to the establishment of the HHE followback program, 
which was fully implemented in October 1999. The goals of the followback pro- 
gram are to "(0 provide feedback to improve the process by which HHEs are 
conducted and improve the worksite-specific outputs of the HHE Program; and 
(2) assess the effectiveness of HHEs in effecting change in the workplace and in 
improving the health of workers" (NIOSH, 2007b:23). Because of financial con- 
straints that limit followback field investigations, followback evaluations consist 
of self-reported satisfaction questions to management and employees in the facil- 
ity served, primarily yielding process rather than outcome information. Question- 
naires are distributed for all field investigation and for 50 percent of those receiv- 
ing technical assistance/consultation. Followback field investigations are con- 
ducted for a very limited number of HHEs per year, resources permitting. This is 
an ongoing process intended to assist the HHE Program with evaluation of cus- 
tomer service, relevance, and impact. In response to another RTI recommenda- 
tion, compendia of HHE reports on lead, tuberculosis, noise, and isocyanates were 
developed (NIOSH, 2001a, 2001b, 1998c, and 2004e, respecfively). 

The 1997 BSC Review was conducted at the request of the NIOSH director 
(NIOSH, 2007b). Recommendations were made related to 

• problems identifying the agenda of the requestor, including labor- 
management difficulties and work organization issues; 

• personnel training needs, especially communicafion; 
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• ihc need to better prioritize HHE requests; 

• doeumentation of HHE impaets, including effectiveness of reeommenda- 
tions; 

• the need to identify emerging hazards, less routine work, and more useful 
summaries of investigations in areas of more extensive experience; and 

• maximizing the ability to accomplish the preceding recommendations 
within the constraints of limited resources. 

The HHE Program responded with significant changes to address the second and 
third of these issues. To improve communication, the program restructured its num- 
bered HHE reports into a uniform format that clearly identifies authorship and re- 
sponsible personnel. The first page of the reports includes a plain-language discussion 
of recommendations for employers and employees, followed by a clear summary. 
Letter reports written by the HHE Program have not been similarly structured, al- 
though authorship and telephone contact information facilitate communication. 

The second major response was to formally introduce a "triage system" to pri- 
oritize incoming requests for HHEs. A mechanism has been developed that allows 
the HHE Program to respond to routine questions through telephone and fax-back 
information sharing when program leadership determines that a field investigation 
would not increase the body of knowledge on a given topic. The triage process 
itself was described to the committee by HHE Program staff The committee 
notes that the criteria for triage, apart from the straightforward assessment of the 
validity of the request as determined by regulation, do not expHcitly outline how 
HHE personnel determine whether field investigations are necessary or further 
distinguish complex investigations or those yielding research opportunities. 

In further response to BSC (NIOSH BSC, 1997) recommendations, the HHE 
Program developed and twice updated a procedures manual used as a reference 
for all program staff Report dissemination, via websites, announcements in 
NIOSH eNews, announcements to all slate epidemiologists and epidemic intelli- 
gence officers through Epi-X, NIOSH alerts, and summary documents on par- 
ticular topics, was enhanced. The ability to triage routine requests by definition 
frees up scarce resources for enhanced focus on emerging issues; however, met- 
rics to assess the impact of these changes were not formally introduced until after 
a subsequent BSC program evaluation in 2006. 

The 2006 evaluation by the BSC was undertaken at the request of the NIOSH 
Director (NIOSH, 2007b). Recommendations include 

• restating HETAB and DRDS missions and increasing the amount of pri- 
oritization of HHE requests (triage) to reduce the number of open projects 
per project officer — the responsibility for responding to routine indoor en- 
vironmental quality (IHQ) requests was recommended to be removed from 
the HHE Program;"^ 



Epi-X is a secure electronic communication network maintained by CDC for public health agen- 
cies nationwide. 
"^ Noie: Thi.s commitlee does not ftjUy agree with this recommendation. 
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• improving the efficiency of processes, prioritization of tasks, and overall 
management of the program, including development of annual goals and 
performance measures, production of annual reports to demonstrate pro- 
gram outcomes, and utilization of outside consultants; 

• identification of the critical mass of staff and specific disciplines necessary 
to perform essential functions; 

• promotion of the HHE Program more widely in an effort to capture more 
emerging issues, and selection of HHEs which will serve program goals; and 

• inclusion of estimated health-related savings as a result of HHE recom- 
mendations. ' ' 

Given the short amount of time since these recommendations were made, the 
HHE Program has not had the opportunity to respond fully to many of them. The 
program has developed annual goals and performance measures, which are dis- 
cussed and evaluated in Chapter 3 of this report. Discussions among HHE Pro- 
gram management have taken place regarding how to proceed with others of these 
recommendations. 



HHE Requests 

As already discussed, the HHE Program receives requests for assistance in 
addressing workplace health issues from stakeholders and applies triage criteria to 
determine whether a field investigation is warranted. The annual number of HHE 
requests has remained relatively fiat for the past 10 years, with an average of ap- 
proximately 370 requests each year (NIOSH, 2007b:35). Figure 2-3 shows the 
distribution of requests made by decade, organized by Standard Industrial Classi- 
fication (SIC).'^ The distribution of HHE requests by sector has changed over 
time. For example, from the 1970s through the 1990s, the manufacturing sector 
accounted for 30 to 40 percent of HHE requests. Today, manufacturing accounts 
for less than 20 percent (NIOSH, 2007b). Table 2-2 shows the frequency of invcs- 
figations compared by NORA industry sectors and hazard (NIOSH, 2007h). ' 



" Note: This committee questions the feasibility of this recommendation. 

'■ The SIC system (replaced by the North American Industry Classiilcation System [NAICS] in 
1997) was developed for use by federal statistical agencies to classify business establishments for 
statistical analysis purposes (http://www. census. gov/epcd/www/naicsdev. htm, accessed April 16, 
2008). HUE Program staff provided data to the commiiiee referring to the SIC system. For the 
sake of consistency, the committee opted to use SIC rather than NAICS. 

'■^ Table 2-2 was derived by the HHE Program in re.sponse to mformation requests from the com- 
mittee (NIOSH, 2007i). The data were drawn from the program's management tracking system, 
developed primarily for record retrieval and not data analyses. Because most HHE requests are 
related to multiple hazards or hazard classes, the HHE Program developed Ihe following strategy 
for the tabulation of hazard class of requests: 

• lEQ — any request which included an lEQ issue, regardless of other issues also present 

• Physical — if not lEQ, and if keywords were present indicating ergonomic, musculoskele- 
tal, radiation, or noise issues 
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Comparing ihe two sectors with the largest number of requests (serviees and 
manufaeturing), manufacturing-seetor requests were more likely to have resulted 
in field investigations than scrviee-seetor requests. 

The HHE Program also reported a change in the nature of requests over time. 
The number of lEQ-related requests, for example, significantly increased follow- 
ing a 1992 CBS evening news segment that encouraged viewers with lEQ prob- 
lems to contact NIOSH (NIOSH, 2007b). More recently, lEQ requests have typi- 
cally been made with regard to nonindustrial worksites, such as government 
buildings, private sector offices, schools, healthcare facilities, and hotels and re- 
sorts (NIOSH, 2007b). 




19B0S 



1990s 



20005 



D Ag & mining ■ Construction □ Manufacturing 
a Transportation ■ Trade Q Health services 

n Other services B Public admin 



FIGURE 2-3 HHE requests by decade by SIC Sector. 
SOURCE: NIOSH (2007b). 



Biological — if not assigned to lEQ or Physical, and keywords were present relating to tu- 
berculosis, histoplasmosis, biosolids or bmcellosis issues 
All Other-^uny other request not assigned to the classes above 
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TABLE 2-2 Frequency of HHE Field Investigations by Sector and Hazard: Fiscal 
Years 1998-2007" 
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1 


n 
(ti) 
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21 

(8) 
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Total 
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, (8) 


1927 


121 
(38) 


315 


37 
(56) 


66 
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(25) 


1623 
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(19) 


3931 



"Tneludes all HHE requests with a completed response as of November 20, 2007. 

''Sector is based on SIC codes or NAlCS codes, depending on year of the request. 

■^See text for an explanation of hazard classes. 

''Percent of HHE requests resulting in a field investigation. 

'T^umber ofHHE requests. 

SOURCE: NIOSH (2007h). 

ACTIVITIES 

Response to Requests 

The HHE Program responds to requests for HHEs or consultation and techni- 
cal assistance in the form of letter or telephone responses and field investigations. 
HHE Program managers generally make decisions regarding the appropriate re- 
sponse to incoming HHE requests. As described earlier, the HHE Program has 
developed a process to prioritize incoming HUE requests and direct resources and 
responses. The decision processes for triage and response are summarized by 
HHE Program staff in Figure 2-4 (NIOSH. 2008d). IIETAB and FSB managers 
and supervisors regularly meet to specify under which of four response categories 
an HHE request may fall (see Box 2-2), following criteria outlined in the HHE 
Program Procedures Manual (NIOSH. 2007g), and summarized in Box 2-3. Con- 
tact with the requestor may be for referral to a different agency, or to obtain addi- 
tional information, assist in the formulation of a valid request, or provide readily 
available information regarding the request. Alternatively, HHE staff may decide 
a site visit is warranted, which may result in a letter report with findings and rec- 
ommendations applicable to the given worksite, or a numbered report that may be 
applicable to other workplaces. 
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NIOSH HHE Decision Process 
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FIGURE 2-4 Flow diagram representing Ihe HHE Program triage process for pri- 
oritizing HHE requests and allocating resources for response. 
SOURCE: NIOSH (2008d). 
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BOX 2-2 

Triage Response Categories 

n order to prioritize HHE requests, the HHE Program will assign each written request for 
assistance in one of the following described categories that drive the program response: 

Category 1. These requests do not meet the criteria for a valid request [per legislation], 
concern issues outside the scope of the HHE Program, and are therefore administratively 
invalid. The HHE Program notifies the requestor and refers the requestor to another 
agency, such as OSHA or a state or local health department. 

Category 2. These are valid or invalid requests for which technical information is supplied 
to the requestor without conducting a field investigation. Examples include well- 
recognized problems with readily apparent solutions, problems that have already been 
adequately evaluated by NIOSH or others. When the request is invalid but the HHE Pro- 
gram believes that a field investigation might be warranted, the requestor is contacted 
quickly and provided information about what constitutes a valid request. 

Category 3. These are valid requests for which a field investigation is necessary to ade- 
quately evaluate the occupational safety and health problem described. 

Category 4. These are valid requests that present a complex problem or an opportunity 
for research. These may take longer than Category 3 requests to complete due to re- 
quired method development or other technical issues. 

SOURCE: NIOSH (2007b). 



BOX 2-3 

HHE Program Triage Criteria 

The HHE Program has created guidelines to categorize and prioritize HHE requests, and 
has summarized them in the HHE Program Procedures Manual (NIOSH, 2007g). Classi- 
fication is based on information provided by the requestors, and is largely dependent on 
the expertise of HHE Program staff and management. The following is excerpted from 
the HHE Program Procedures Manual. 

HHE Categorization Strategy 

1 . Valid request? If the request is Invalid, the requestor will be given the criteria for a valid 
request and given the opportunity to resubmit the updated request. 

2. Does the request relate to a NIOSH special initiative or research project? For example, 
violence In the workplace, agricultural or construction industries would be placed in Cate- 
gory III or IV. 

3. What type of exposures are present in the workplace? Unfortunately, the information 
included in the original request is frequently incomplete. Therefore, requests from industries 
with known exposure to extremely hazardous substances should be placed into Category III. 
In addition, new industries, processes, or exposures not previously evaluated by the HHE 
Program, NIOSH in general, or other occupational safety and health professionals should be 
placed into Category III or IV. On the other hand, requests for compliance or routine 
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monitoring would be place into Category II with referral suggestions {e.g. OSHA). Finally, 
processes previously evaluated by the HHE Program and found to have minimal opportunity 
for hazardous exposures should be categorized as either Category II or III. 

4. Are adverse health effects being reported? Like the exposure information, information 
about alleged health effects is frequently misleading. Considerations used to categorize 
requests based on health effects are the 1) type of condition, 2) severity of the condition, 
3) reversibility of the condition, and 4) amenability to control measures. If the condition 
has been evaluated by a physician, has a diagnosis been made? Severe symptoms 
causing hospitalization should be categorized into III or IV. Mild symptoms should be 
categorized into II or III. Previously unrecognized health effects from common exposures, 
or occupational groups not known to be at risk for a particular health effect should also 
categorized as III or IV. Requests intended only to provide medical or industrial hygiene 
evidence for a pending legal action should be placed in Category II. 

5. Is the workforce unique? For example, is the workforce composed of only women, only 
minorities, migrant labor, etc. If yes. Category III should be considered. 

Criteria for Prioritizing HHE Requests in Category HI 

1 . Does the request relate to a NIOSH special initiative or research project? For example, 
violence in the workplace, agricultural or construction industries. 

2. Can the evaluation be done by someone else? Examples: Companies can hire private- 
sector consultants. Government agencies can use Federal Occupational Health (FOH); 
some agencies have their own health and safety staff. OSHA requesters can consult with 
their national Technical Assistance Directorate in Washington DC. Some state health de- 
partments have the resources to investigate. In some cases, an OSHA technical consulta- 
tion {for management) or inspection {employee) may be more appropriate than an HHE. 

3. Who is the requestor? Employee, small business, and union requests will be given a 
higher priority given Iheir resource and access limitations. 

4. What types of exposures are present in the workplace? Requests from industries with 
known exposure to extremely hazardous substances should be given a higher priority 
than other Category III requests. In addition, new industries, processes, or exposures not 
previously evaluated by the HHE Program. NIOSH In general, or other occupational 
safety and health professionals should be ranked higher. Industries or processes previ- 
ously evaluated by the HHE Program should be ranked lower. 

5. Are adverse health effects being reported? Considerations used to prioritize requests 
based on health effects are similar to those used to categorize requests. Therefore, the 
1) type of condition, 2) severity of the condition, 3) reversibility of the condition, and 4) 
amenability of control measures are important to consider. Has the condition caused the 
affected employees to seek medical care? Severe symptoms causing hospitalization 
should be given high priority. Mild symptoms should be categorized into It or III. Previ- 
ously unrecognized health effects from common exposures, or occupational groups not 
known to be at risk for a particular health effect should be given higher priority. Requests 
intended only to provide medical or industrial hygiene evidence for a pending legal de- 
termination should have a low priority, 

6. Geographic, In times of critical shortage of travel funds, is the request close to Cincinnati, 
Morgantown. or one of our regional offices, therefore incurring minimal travel expenses? 

SOURCE: NIOSH 2007c. 
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The number of field investigations has decHncd during the past 10 years, from 
126 in 1997 to 58 in 2006 (NIOSH, 2007b). HHE Program staff explained the de- 
crease as due, in part, to the program's ability to respond without field investiga- 
tions to requests for which there are well-established control solutions. Additional 
factors attributed to the changing response pattern include the decreased size of 
HHE Program staff, decreased discretionary funding, increased complexity of 
field investigations, and increased demands for staff involvement in activities 
other than traditional HHEs. These activities include emergency response and 
preparedness, participation on agency and expert committees, document devel- 
opment and review, international technical assistance, and mentoring and training 
of non-NIOSH occupational health and safety professionals (NIOSH. 2007b). 

The HHE Program may conduct a followback survey at a facility where an 
HHE has been conducted. The surveys arc distributed among management and 
employees, and are designed to obtain feedback to improve the HHE process and 
assess the impact of the HHE in the workplace. 



Emergency Response 

The HHE Program's responsibilities in response to disasters and emergency pre- 
paredness have changed dramatically since the terrorist attacks on September 1 1 , 
2001 (9/11), and the program's role is still evolving. Because of HHE Program staff 
experience in assembling rapid field response, they provided leadership for NIOSH 
activities in the aftermath of the 9/11 attacks on the World Trade Center and for re- 
sponse to the subsequent anthrax contamination through the U.S. postal system. Fif- 
teen HHE Program staff, along with other NIOSH stall, responded to the immediate 
protection needs of rescue and recovery workers in New York City. Such workers 
included firefighters, law enforcement personnel, emergency medical services, con- 
struction and demolition trades, health and safety personnel, volunteer workers, and 
local, state, and federal agency workers. The HHE Program provided technical assis- 
tance to local, state, and federal agencies related to personal protective equipment; 
assisted with air sampling to characterize the complex environment; and conducted 
several specific HHEs (NIOSH, 2007b). 

HHE Program staff described the training and resource development involved 
in preparing for their role in emergency response settings. The HHE Program pro- 
vides staff training, personal protective equipment and fit testing, and appropriate 
medical evaluations, using an all-hazards model. The model includes biological, 
chemical, explosives, and radiological events, as well as natural disasters, includ- 
ing hurricanes, earthquakes, and wildland fires. The HHE Program participates in 
tabletop exercises that include some staff training with partners in Top Officials 
(TOPOFF), a terrorism preparedness exercise involving officials from all levels of 
government, representatives from the international community, and the private 
sector.''^ 



'■^ Department ot" Emergency Management and Homeland Security (http://\v"ww. cl.gov/demlis/ 
cwp/view.asp?a=2n 1 7&q=290966&pp= 1 2&n= 1 , accessed .luly 8, 2008). 
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Before 9/11, the HHE Program responded to emergencies and managed the 
activities of deployed program staff Subsequently, NIOSH and HHE Program 
roles in response have been prescribed by the National Response Framework 
(U.S. Department of Homeland Security, 2008).'^ The HHE Program docs not 
serve as the lead agency, but has a defined responsibility. The program is one of 
the biggest contributors of technical leadership, but the organizational structure 
falls under the NIOSH OD. HHE Program staff members have often been asked 
to serve as team leaders because of their extensive field experience in difficult 
situations, for example, in the response to Hurricane Katrina in 2005. Many of the 
people staffing the CDC operations center were HHE Program staff 

After 9/11, the HHE Program received substantial funding for a truck and 
trailer for field deployment, and is responsible for its maintenance and use. The 
program also received funding to construct a garage to house the vehicle, and for 
staging purposes. The equipment is housed at the program's Cincinnati facility. 

Pre- and post-deployment medical follow-up have been provided for program 
staff deployed in emergency response activities. Telephone guidance and other 
resources have been made available to assist staff with mental health issues when 
considered necessary (NIOSfl, 2008a). 



OUTPUTS 

HHE Program outputs include official numbered reports and less formal letter 
reports in response to requests for HHEs and technical assistance. The program 
also produces articles for peer-reviewed publications, website content, and pres- 
entations at professional, trade, and agency conferences and meetings. The pri- 
mary means by which the program communicates its findings is through num- 
bered HHE reports. The current format of numbered reports includes a page of 
HHE "highlights" written in lay terms, which includes information about the HHE 
request, program activities and findings, and buUctcd lists of what managers and 
employees can do to minimize investigated hazards. A more technical summary 
follows the highlights section. The body of the report then provides background, 
investigation methodologies and criteria, results, discussion, conclusions, recom- 
mendations, and a list of references. Test results are also made available. 

Technical assistance and letter reports can be very similar in scope to num- 
bered HHE reports, or may be much more focused and answer specific questions 
or provide referrals as necessary. 

Between 1996 and 2006, the HHE Program produced 495 numbered HHE re- 
ports, 503 letter reports for other field investigations, and 1,999 technical assis- 
tance or consultation letter reports (NIOSH, 2007b). The committee reviewed 
several numbered and letter reports provided by the HHE Program, as well as 



The National Response Plan was replaced by the National Response Framework, effective 
March 22, 2008. The National Response Framework presents the guiding principles that enable all 
response partners to prepare for and provide a unified national response to disasters and emergen- 
cies. It establishes a comprehensive, national, all-hazards approach to domestic incident response 
(http://www.dhs.gov/xprcpresp/conimittees/editorial 0566-shtm, accessed July 8, 2008). 
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several HHE reports obtained elsewhere. These and other HHE Program outputs 
are discussed and evaluated at greater length in Chapters 3 and 4. 
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EVALUATION OF RELEVANCE 

As discussed in Chapter 1, a relevant program can be described as an inte- 
grated program involving interrelated surveillance, research, and transfer activi- 
ties. The relevance of its activities can be determined by the severity or frequency 
of hazards addressed and the number of people at risk, the extent to which the 
program addresses gendcr-rclated issues and those of underscrved populations 
(see Box 1-1 for definition), and the extent to which the health and safety needs of 
small businesses arc addressed. Program structure and content must also lend to 
program relevance. 

The Health Hazard Evaluation (HHE) Program primarily serves a legislated, 
public-health practice mission, and its relevance must be evaluated in that context. 
HHEs address issues that, by definition, arc in the early stages of investigation 
and intervention. One important function of an HHE is to provide preliminary 
data thai may lead to a more extensive research program. The mandate of the pro- 
gram, however, requires it to respond to requests, whether or not more in-depth 
research may follow. Within this constraint, the committee believes the program 
can elicit and prioritize requests to fit into an interrelated surveillance, research, 
and transfer matrix involving the rest of the National Institute for Occupational 
Safety and Health (NIOSH). One of the challenges faced by the HHE Program is 
weighing the need to be responsive to individual requests with the need to de- 
velop strategic approaches to identifying emerging health threats in the work- 
place. A carcfiil balance serves both functions. Through response to requests the 
program may identify emerging issues, and identifying emerging issues helps in 
priority-setting among investigations. 

To assess program relevance, the committee begins this chapter with a review 
of major inputs to the program, including the strategic planning and stakeholder 
processes, and considers whether resources have been appropriately allocated and 
used to meet strategic needs. The chapter continues with an evaluation of the 
HHE Program's activities and outputs, a discussion of how the program handles 
emerging issues, and then a discussion of the relevance of the program's emer- 
gency response activities. The committee's findings related to each of these topics 
arc highlighted in bold. Recommendations related to the improvement of program 
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relevance are found within the body of the text throughout the chapter, and are 
organized and summarized in Chapter 5. The final section of this chapter provides 
the committee's rating for relevance. 



RELEVANCE OF INPUTS (PLANNING AND RESOURCES) 

Strategic Planning 

Finding I: The HHE Program strategic plan is highly relevant to the pro- 
gram mission. 

The HHE Program's strategic and intermediate goals are summarized in Table 
3-1. These were developed by the HHE Program in response to a recommendation 
by the NIOSH Board of Scientific Counselors (BSC) to "improve the efficiency 
of processes, prioritization of tasks and overall management of the program [in- 
cluding to . . . djevelop annual goals and measurable objectives regarding the work 
products, priorities and work processes of the HHE Program that are practical, 
cost effective, and consistent with resources" (NIOSH BSC, 2006). The commit- 
tee finds that the HHE Program has made a serious effort to respond to this rec- 
ommendation. The goals are well targeted and relevant, and can be measured by 
the number and types of HHE requests received, by the number of reports and 
field investigations conducted related to each of these goals, and their effects as 
determined through followback surveys (described in Chapter 2). Appropriate 
performance measures for each intermediate goal have been chosen, given the 
limited resources of the program. The committee believes, however, that with ad- 
ditional resources, performance measures could be made more specific and ambi- 
tious. There is no indication that surveillance or workers' compensation data were 
used to fonnulatc the HHE Program's strategic plan. Given more resources, on- 
site follow-up investigations to assess the impact of interventions would help the 
program assess and revise its goals and performance measures as necessary. The 
committee comments on individual performance goals and measures in Table 3-1. 

Recommendation: Conduct regular assessments of performance measures to 
determine whether available resources allow more ambitious goals. 
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TABLE 3-1 HHE Program Strategic and Intermediate Goals, Performance Measures, and Committee Comments 
Strategic goal 1: Prevent occupational illnesses through reduced exposure to workplace hazards. 



Intermediate 

Goals 



Performance Measures 



Committee Comments 



1.1. Increase the 
number of HHE 

requests concern- 
ing important oc- 
cupational health 
problems 



1.1.1. 



1.1.2. 

1.1.2.1. 



Annual goal: Conduct a targeted markeling campaign 

FY 2007: Award one contract for a customer survey 
FY 2007: Stimulate two valid HHE requests through outreach to 
physicians in the Cincinnati area 
Annual goal; Enhance awareness of the HHE Program 

FY 2007: increase the percentage of national union websites that 
include a link lo the HHE Program website (obtain baseline 
data) 
1 . 1 .2.2- FY 2007: Increase the percentage of state OSHA websites that 
include a link to the HHE Program website {obtain baseline data 
for this) 

FY 2007: Establish links to the HHE Program website on the 
website of three government agencies or trade associations pro- 
viding services to small business 

FY 2007: In the 16 states in which an HHE was announced in 
Epi-X in FY 200^1—2006, assess awareness of the HHE Program 
among state epidemiologists 



1.1,2.3. 



1,1.2.4. 



The committee endorses this intermediate goal, which is 
the core of the HHE Program mission. In addition to en- 
hancing awareness, appropriate performance measures 
include establishing partnerships lo develop valid re- 
quests from small businesses and underserved worker 
populations. Partnerships with nonprofit organizations, 
community' development organizations, and worker cen- 
ters could be e.Kplored, and specific metrics concerning 
requests could be generated and tracked. The use of in- 
temational, national, state, and local surveillance data 
would facilitate identifying and focusing resources in 
areas of emerging need and underserved populations. 
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1.2. Facilitate im- 
plementation of 
recommendations 
at facilities investi- 
gated through the 
HHE Program 



1.2.1. Annual goal; Provide a service that is valued by HHE partici- 
pants 

1.2.1.1. FY 2007: 80% of respondents to the HHE followback survey re- 
port that ihe NIOSH evaluation was "good" or "'excellent" one 
year after the report was issued (baseline: 82% for FY 2001- 
2005). " 

1 .2.2. Annual goal: Provide a service that results in improved 
workplace conditions 

1,2,2,1. FY 2007: 80% of respondents to the HH E followback survey re- 
port that workplace conditions have improved since the NIOSH 
evaluation (baseline: 81% for FY 2001-2005)." 

1 .2.3. Annual goal: Produce timely reports for HHE requests ad- 
dressed with a field investigation 

1.2.3.1. FY 2007: Complete 50% of reports within 6 months of the last 
site visit (baseline: 36% for investigations with a last visit in FY 
2005)." 

1.2.4. Annual goal: Produce useful reports for HHE requests ad- 
dressed wirh a Field investigation 

1.2.4.1. FY 2007: 95% of respondents to the HHE followback survey re- 
port that the Recommendations section of the report was "very or 
somewhat useful" to them.(baseline: 94% for FY 2001-2005). " 
1.2.4.2. FY 2007: 95% of employees responding to the HHE fol- 
lowback survey report that the Highlights section of the report 
was ''very or somewhat useful" to them (baseline: 96% for FY 
2001-2005)." 

1.2.5. Annual goal; Provide continuing technical assistance after the 
issuance of the final report 

1 .2.5.1 . FY 2007: Make onsite, post-investigation presentations for two 
HHEs 

1.2.5.2. FY 2007: Develop a partnership with one facility to assist in 
implementation and evaluation of recommendations 

1.2.6. Annual goal: Assess implementation of HHE recommendations 
through followback surveys 

1.2.6.1. FY 2007: Carry out a mailed survey regarding recommendations I 
year after issuing a final report for 100%) of on-site HHEs 

1.2.6.2. FY 2007: Successfully obtain information from 70% of partici- 
pants in the mailed survey 

1.2.6.3. FY 2007: Complete at least four onsite followback stirveys 



While the committee agrees there is a need for continued 
commitment to process evaluation, return site visits chat 
offer the ability to provide objective evaluation of impact 
are critically important, though adequate resource may 
need to be obtained. 

The committee notes a decrease in field investigations, 
which may at least partially explain the decrease in re- 
sponse time for investigations that are being conducted. 
The timeliness of the responses to HHE requests is criti- 
cal to the success of the program. With the diminishing 
number of HHE requests, the program should be able to 
set more timely response goals. 

Follow-up with technical assistance after report issuance 
should be carefully evaluated for knowledge transfer and 
exchange to other relevant facilities. 
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1.3. Provide appro- 
priate technical 
assistance for HHE 
requests addressed 
without a field visit 


1.3.1. Annual goal: Provide timely informational letters 

1.3.1.1 , FY 2007: Complete 90% of letters within 6 months of the request 
(baseline: 87% for FY 2005 requests)," 

1.3.2. Annual goal: Provide helpful informational letters 

1 ,3.2.1 FY 2007: 65% of HHE requesters completing a followack survey 
report that the letter was helpful {baseline: 60% for FY 2001- 
2005)." 

1,3.2.2, FY 2007: 65% of IIHE requesters completing a foUowback sur- 
vey report that they shared information in the letter with others at 
the worksite (baseline: 59% for FY 200 1-2005). ^ 

1.3.3. Annual goal: Develop standard informational materials for 
common concerns 

1,3.3,1, FY 2007: Two draft lEQ quality NIOSH-numbered docu- 
ments will be submitted for external peer review 


Performance measure 1.3.3 would greatly free up re- 
sources while informing a wide range of workplaces 
about ways to reduce poor indoor environmental quality 
(lEQ). It should be possible to produce similar educa- 
tional materials for other areas of high-frequency requests 
in which the HHE Program has considerable experience. 


1.4. Employees and 
employers at facili- 
ties not investigated 
through the HHE 
Program arc aware 
of hazards identi- 
fied and controls 
recommended by 
HHE Program in- 
vestigators 


1.4.1. Annual goal: Enhance the dissemination of HHE reports 

1.4. 1 .1. FY 2007: Announce all HUE reports in eNews and Epi-X within 
2 months of distributing to the investigated facility 

1 .4. 1 .2. FY 2007: Post HHE reports on the NIOSH website within two 
months of distributing to the investigated facility 

1.4.2. Annual goal: Develop communication products based on HHEs 
(e.g., NIOSH-numbercd documents^ trade publications, and 
presentations) 

1 .4.2.1. FY 2007: Prepare and disseminate 1-2 new written products 

1.4.2.2. FY 2007: Present HHE findings at 2-3 trade meetings 

1.4.2.3. FY 2007: Prepare and disseminate an annua! report of highlights 
from the HHE Program 


This intermediate goal should further increase program 
relevance. Additional avenues for disseminating infomia- 
tion may include video-sharing websites or public service 
announcements. County agriculture extension services 
may be used to disseminate agriculture- related findings. 
Similarly, some states have engineering extension ser- 
vices to work with small manufacturing facilities and 
economic development councils. 


1.5. Professional 
practices, guide- 
lines, policies, stan- 
dards, and regula- 
tions arc influenced 
by information 
generated from the 
HHE Program 


1.5,1. .Annual goal: Transfer information generated from the HHE 
Program to other occupational safety and health professionals 

1.5.1.1. FY 2007: Submit four to six peer-reviewed manuscripts based on 
HHEs 

1.5.1.2. FY 2007; Make 20 to 25 presentations based on HHE Program 
work at scientific or agency meetings 

1.5.1.3. FY 2007: Participate as requested in agency workgroups 


The committee applauds the HHE Program's recognition 
of the critical role it can play in knowledge transfer, and 
encourages an increased focus on knowledge exchange to 
increase relevance in identifying emerging issues. 
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Strategic goal 2: Promote occupatioml safety and health researchi^n eniergins issues. . -• v ■ aii^? / 


Intermediate 
Goals 


Performance Measures 


Committee Comments 


2.1. Customers will 

have information 
about emerging 
issues 


2.1.1. Annual goal: Use the internal HHE review process and HHE 
project officer surveys to identify emerging issues 

2.1.1 .1. FY 2007: Prepare and disseminate an annual list of new 
findings from HHEs 


The approach described could be augmented by develop- 
ing a tickler file of unusual requests for which no clear 
findings have been made. These could be freshly re- 
viewed periodically in a process that includes intramural 
and extramural scientists to identify new patterns. 




2. 1 .2. Annual goal: Transfer information regarding emerging issues 
to appropriate internal and external partners. 

2.1.2. 1. FY 2007: Complete and accurate information regarding new 
HHE requests and closed projects is entered in the HHE database 

2. 1 .2.2. FY 2007: Prepare and disseminate annual HHE summary data to 
National Occupational Research Agenda (NORA) research coun- 
cils for the manufacturing, construction, service, and healthcare 
and social assistance sectors. 

2.1 .2.3. FY 2007: Provide timely summary information generated 
from the HHE Program to all NORA sectors. 


Interaction with NORA sector councils will increase rec- 
ognition of the HHE Program within these industries as 
well as the HHE Program's understanding of potential 
hazards within each industry. Sectors chosen by the HHE 
Program are certainly relevant, but the program could 
also consider being represented on the NORA transporta- 
tion council. This industry has high injury and illness 
rates, particularly within trucking, and few HHEs have 
been conducted (n = 3, with the last one in 1989). 


Strategic goal 3: Protect the health and safety of workers during public health emergencies. 


Intermediate 
Goals 


Performance Measures 


Committee Comments 


3.1. Partners and 
customers have the 
information they 
need regarding 
high-priority occu- 
pational health 
issues likely to arise 
during public 
health emergencies 


3.1.1. Annual goal: Develop and disseminate information anticipating 
likely chemical., biological, or radiological threats 

3.1.1.1. FY 2007: Complete 100% of tasks related to pandemic tlu plan- 
ning according lo schedule 


The HHE Program has been effective in this arena and 
should communicate the importance of this role to ensure 
resources are available to continue these activities. 


3.2. HHE Program 
personnel respond 
appropriately to 
requests for assis- 
tance 


3.2.1. Annual goal: Maintain adequate numbers of trained and certi- 
fied staff 

3.2. l.l , FY 2007: A minimum of 25 personnel are enrolled in a respira- 
tory protection program, maintain Hazardous Waste Operations 
and Emergency Response Standard (FIAZWOPER) certification, 
complete incident command system training, and are aware of in- 
formation resources regarding specific potential hazards 

3.2.1.2. FY 2007: Assurance is received from the Office of Health and 

Safety that HHE emergency responders have appropriate medical 
oversight 


Emphasis should be placed on recognizing policy impli- 
cations of lIHEs in this arena. Explicit training should be 
given on how to recognize and appropriately seek guid- 
ance on handling HHE requests with broader policy im- 
plications. 
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46 THE HEALTUI lAZARD EVALUATION PROGRAM AT NIOSU 

Use of Resources 

The Framework Document {Appendix A) directs the committee to consider 
how planning, production, and other input data support and promote program 
goals and activities, and to determine whether input is obtained from stakeholders, 
including from underscrvcd populations and small companies. Planning inputs 
include surveillance data; advice from advisory committees, National Occupa- 
tional Research Agenda (NORA) research partners, and other stakeholders; re- 
ports from the Fatality Assessment Control and Evaluation (FACE) program; and 
extramural health-outcome, exposure assessment, or similar data from federal, 
state, and other program partners. Production inputs, including budget, staff, fa- 
cilities, and management, play major roles in the program. 

Finding 2: For the most part, the HHE Program has judiciously used its 
resources to meet its mission in the face of the changing economy, the 
changing nature of HHE requests, and increased responsibilities related 
to emergency response. 

HHE Program activities have been modified over the past 10 years because of 
events such as the September 1 1, 2001 (9/11) attacks, specific HHE topics such as 
diacetyl, the influence of the NORA process, and response to program evalua- 
tions. The 1997 BSC evaluation {NIOSH BSC, 1997) prompted the formalization 
of the HHE Program triage process, ' described in Chapter 2, and the policy man- 
ual, which have resulted in improved program efficiency and other improvements. 



Health Hazard Evaluation Requests and Resource Allocations 

Figure 3-1 shows the change in the number of HHE requests and program re- 
sponses between 1997 and 2006. The number of requests during this period peaked 
in 2001 at approximately 580. At the end of the timeframe, HHE requests declined 
to approximately 390, which is only approximately 15 percent more than in 1997. 
By the end of the period, more responses to HHE requests took the form of written 
information and telephone communication rather than field investigations. These 
forms of follow-up were specifically developed in response to extcmal reviews and 
indicate HHE Program success in redirecting resources to provide appropriate in- 
formation in response to routine questions. However, this increased proportion of 
routine requests may also suggest the HHE Program may now have the opportunity 
to conduct more outreach and encourage more requests (for example, those that 
meet strategic needs or represent new or emerging hazards) which would require 
site visits (assuming sufficient funding is available to conduct them). 



' Triage is the system for allocating scarce resources lo cases or problems based on a priority 
scheme, especially in emergency medicine and public health. The term comes from battletleld 
medicine where the wounded were separated into three categories — those who were likely to die 
even with care, those who were likely to rccovej' even without care, and those for whom care 
would be effective. 
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FIGURE 3-1 Distribution of HHE requests by response category. 

NOTE: Category i and 2 requests are combined under technical assistance/consultation. Category 

3 and 4 requests are combined under field investigations {see Box 2-2 for a description of the HHE 

request categories and responses). 

SOURCE; NIOSH (2007b). 

The committee the HHE Program provide information regarding all HHE re- 
quests, including types of requestor, hazard and sector involved, and the types of 
responses. Tables 3-2 and 3-3 were derived by the HHE Program in response 
(NIOSH, 2007h). The data were drawn from the program's management tracking 
system, which was developed primarily for record retrieval and not data analyses. 
Because most HHE requests are related to multiple hazards or hazard classes, the HHE 
Program developed the following strategy for tabulation of hazard class of requests: 

lEQ — any request that included an lEQ issue, regardless of other issues 
also present 

Physical— if not lEQ, and if keywords were present indicating ergonomic, 
musculoskeletal, radiation, or noise issues 

Biological — if not assigned to lEQ or Physical, and keywords were pre- 
sent relating to tuberculosis, histoplasmosis, biosolids or brucellosis issues 
All Other — any other request not assigned to the classes above 

In its response to the committee, the HHE Program recognized the limitation of 
this methodology. 

Table 3-2 lists the type of HHE response by source of request, hazard, and 
percentages of resulting field investigations by requestor type between 1998 and 
2007. During this time, approximately half the requests were related to lEQ is- 
sues, and were largely addressed without field investigations. Among the five sec- 
tors with more than 100 requests, the correlation between the percentage of field 
investigation responses and the percentage of lEQ requests is -0.82. There were 
very few requests related to biological issues, thus it was appropriate that the 
greatest proportion of requests resulting in field investigations occurred in this 
area. Many requests in the physical hazards category are likely related to muscu- 
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loskeletal hazards, which is appropriate given the magnitude of the problem in 
most industries. The committee wanted to explore trends in requests, both with 
and without consideration of lEQ requests, to determine if those requests poten- 
tially skew the data. The HHE Program databases could not provide the data in a 
manner that would allow more robust analysis. 

TABLE 3-2 Frequency of HHE Field Investigations by Source of Request and Hazard Class: 
Fiscal Years 1998-2007" 

Source of Request (Requestor) 



W ■l'/-!*r/l 


EmpI 


.oyee 


Govemmenl 


Management 


Union 


Joint 


Other 


Total 


J la^rUi M 
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* N "" " 
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N 
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N 
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N '■ Total 




(%)'' 
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. (%)■' 
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(%)■' N^ 


{%f 


N' 


, (%)'' 


N'' 


. (%)" 


N^ 


(%f N' 


Biological 


1 
(8) 


12 


26 
(87) 


30 


8 
(53) 15 
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(40) 
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62 
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51 


49 ' 

■■ lit, 


24 


189 
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214 
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(13) 




(54) 




(45) 


(39) 




(73) 




(4) : 




(25) 


Total 


200 




146 




218 


127 




27 




1 i 




729 






2319 




266 


,^r,-. ■ 560 




455 




42 




290 


3932 




(9) 




(55) 




_ .(39) 1 . 


(28) 




(64) 




(4) 




(9 



"Includes all HUE requests with a completed response as of November 28, 2007, 

''This group may include some HHE requests more appropriately classified as management due to a change in 

coding over lime. 

'See text for an explanation of hazard classes. 

''Number (percent) of HHE requests resulting in a field investigation. 

'^Jumber of HHE requests. 

SOURCE: NIOSH (2007h). 

The effect of the changing economy can be seen in Figure 2-3, which indi- 
cates a greater than 50 percent decline in the number of requests from the manu- 
facturing sector since the 1990s. One might expect this change to allow the pro- 
gram greater flexibility to more efficiently respond to the increasing number of 
HHE requests in health services, trade, and transportation, and to conduct unex- 
pected emergency response activities. Telephone, fax, and written responses to 
HHE requests have increased in the past 10 years (NIOSH, 2007b). The percent- 
age of field investigations generated by lEQ and service-sector requests is very 
low compared those from manufacturing (see Table 2-2). 

To facilitate efficiency when responding to requests, HHE Program staff re- 
views available reports from other parties who have investigated the issue or site 
and provides opinions regarding the methods, data interpretation, and appropri- 
ateness of conclusions and recommendations. HHE Program staff may then gather 
additional information from employer and employee representatives during the 
triage process to determine the request category and HHE Program response. 
HHE Program staff consult with other NIOSH scientists, review the scientific lit- 
erature, and prepare written responses summarizing their activities. These steps 
improve efficiency and provide consistency in HHE outputs. 

The committee notes that the percentage of field investigations lends to vary 
by requestor type. For example. Table 3-2 indicates there were large numbers of 
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employee requests (n = 2,319), of which 9 perccnl (n = 200) resulted in field in- 
vestigations, whereas 39 percent of the far fewer management requests (n = 560) 
resulted in field investigations. More than 1,300 of the employee requests were 
for lEQ issues. As described below, NIOSH appears to have used appropriate dis- 
cretion in providing extensive information on control measures to the vast major- 
ity of lEQ requesters in the form of technical assistance and consultations. 

Table 3-3 addresses the type of HHE response by source and sector. There 
were very few requests from agriculture and mining. The proportion of requests 
resulting in field investigations was less for healthcare and social assistance (13 
percent) and services (15 percent) than for most other sectors. While this is likely 
related to lEQ requests identified in Table 3-2, these sectors may be growing, and 
also reflect a higher proportion of underscrved populations, again potentially indi- 
cating the need for the program to assist stakeholders in developing meaningful 
and valid requests. Similarly, industries generating few requests might benefit 
from HHE Program exploration into the reasons so few requests are made fol- 
lowed by appropriately directed outreach activities. 

TABLE 3-3 Frequency of HHE Field Investigations by Source of Request and Sector: Fiscal Years 
1998-2007'^ 

Source of Request (Requestor) 



Sector'' 

Agriculture, 
Forestry, & 
Fishing 
Construction 

Healthcare 
& Social 

Assistance 
Manufactur- 
ing 
Mining 

Services 
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(34) 


35 


289 


33 
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5 
(39) 
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(15) 
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"Includes all HUE requesis with a completed response as of November 20, 2007. 

'"Sector is based on Standard Industrial Classification codes or North American Industry Classificalion System codes. 

depending on the year of the request. 

'This group may include some HUE requests more appropriately assigned to the management group due to a change in 

coding over time. 

"Percent of HHE requesis resulting in a field investigation. 

'"Numberof HIIF requests. 

SOURCE: NIOSH (2O07h). 
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As noted in Table 3-1, the committee endorses the HHE Program's intermedi- 
ate goal of increasing the number of HHE requests concerning important occupa- 
tional health problems (intermediate goal 1.1). The committee agrees that an in- 
crease in valid requests will increase the scope and penetration of the HHE Pro- 
gram into more industries and allow better transfer of information to more work- 
ers, including those from underserved populations and employed by small busi- 
nesses. During discussions with the committee, HHE Program staff appeared 
aware that with an increase in the number of requests comes the potential for in- 
creased constraints on resources. 



Program Resources and Emergency Response 

In the last decade, the HHE Program has received a number of requests that 
involved assessing risks, measuring exposures, and providing guidance to workers 
and the general public during disasters (NIOSH, 2007b). These assignments often 
involved the on-site presence of HHE staff working in concert with partners from 
the Federal Emergency Management Agency (FEMA), the Department of Health 
and Human Services Department of Emergency Response, the Center for Disease 
Control and Prevention (CDC) National Center for Environmental Health, the 
CDC Director's Emergency Operations Center (DEOC), and several global agen- 
cies and manufacturers. Response to domestic disasters such as Hurricanes 
Katrina, Floyd, and Isabel, and massive flooding required mobilization of up to 18 
HHE Program staff rcsponders, smaller numbers of public health professionals for 
onsite investigations, or long-distance support from HHE Program staff through 
conference calls (NIOSH, 2008b). HHE Program staff also responded to events 
such as 9/1 1 and subsequent anthrax investigations. 

Emergency response activities create competing demands on limited resources 
within the HHE Program. Routine program operations continued during these un- 
planned activities, but significant coordination and additional outside funding 
were required to maintain continuity. Requests for assistance during disasters 
have so significantly impacted the workload of HHE Program staff that the BSC 
was prompted to recommend major process changes to improve efficiency 
(NIOSH BSC, 1997). The triage process and a policy manual developed in re- 
sponse to the evaluation have created tlcxibility and efficiency to effectively re- 
spond to public emergencies while maintaining normal program operations. More 
discussion on HHE Program emergency response activities is found in a later sec- 
tion of this chapter (Finding 9). 

Recommendation: Continue to provide guidance and recommendations dur- 
ing public health emergencies. The HHE Program would benefit from the 
development of a mechanism to reduce the impact of emergency response 
activities on routine program functions. 
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Expertise 

With the exception of pulmonologists on staff in the Field Studies Branch 
(FSB), the HHE Program, out of necessity, is staffed primarily with generalists 
(N(OSH, 2007b) and must rely on resources beyond the HHE Program for spe- 
cialized expertise. The HHE Program informed the committee how necessary ex- 
pertise is identified during the triage process, and indicated that many areas of ex- 
pertise are readily available through existing, relatively informal arrangements 
within and external to NIOSH. It was not made clear how certain areas of exper- 
tise, such as dermatology or clinical toxicology, are accessed, or how HHE staff 
acquire information in such areas as health services or intervention effectiveness 
research. Nor was it made clear whether there was explicit understanding of the 
economic environment of a worksite — important for assessing the economic fea- 
sibility of recommended control solutions." The HHE Program's ability to utilize 
expertise from other parts of NIOSH could be enhanced by a formal mechanism 
to contract externally for needed clinical, toxicologic, or other specific expertise, 
especially as resources become limited and experienced individuals retire. For- 
malized relationships for cooperation may be helpful, but should be careililly ne- 
gotiated to ensure continued flexibility when people, expertise, or analytical ser- 
vices beyond the terms negotiated are needed. 

A program that addresses the wide array of issues as docs the HHE Program, 
including in emergency situations, would not be successful without residing in 
and having access to the expertise in NIOSH. A review of the authorship of HHE 
reports and subsequent peer-reviewed journal articles indicates that the HHE Pro- 
gram uses expertise from elsewhere in NIOSH for a wide variety of occupational 
health issues, and that successful collaborations have resulted in peer-reviewed 
publications. However, as in many other agencies, programs, and institutes that 
employ public health professionals, many NIOSH senior professionals are ap- 
proaching retirement. The HHE Program and NIOSH could prepare for this shift 
through succession planning, contemplated rcassignmcnts, and identification of 
key resources needed in the near future (see the committee's discussion of the 
training of new professionals in Chapter 4). 

Finding 3: The HHE Program has not made sufficient use of available 
surveillance data to assist in targeting field investigations to recognize 
previously unknown hazards, or to identify new or increased hazards 
caused by changes in the workforce and work environment. 

The committee was informed that the HHE Program uses traditional methods, 
such as results from investigations, literature reviews, case files or other internal 



" The BSC recommended the IIHE Program include estimates of" costs and savings associated with 
implementing HHE recommendations (NIOSH BSC, 2006). The committee agrees that feasibility 
is an important consideration when making recommendations. The expertise needed to provide 
cost estimates, however, does not reside within the program and providing such estimates could 
delay dissemination of results and potentially detract from the HHE Program mission. 
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documents, and contacts within the scientific community to identify emerging, 
previously unknown, or increased hazards in the workplace. 

There is no indication that surveillance or workers" compensation data are 
used to formulate the HHE Program's strategic plan, or that intervention effec- 
tiveness data are examined prior to making recommendations. The program's 
strategic plan may have been influenced by surveillance data, given the extent 
NORA and the overarching NIOSH strategic goals may be influenced by surveil- 
lance data. There is also little evidence that the HHE Program conducts environ- 
mental scans and routinely monitors various state and federal databases (for ex- 
ample, OSHA's Integrated Management Information System database [IMIS] or 
Bureau of Labor Statistics [BLS] databases), reports, or websites on which sur- 
veillance information may be available to identify potential emerging hazards (see 
Box 3-1). An environmental scan is more than a review of injury and illness sta- 
tistics. It takes into account societal and economic changes and pressures, demo- 
graphic changes, and the ways industry uses new technology and the workforce, 
as well as how government action and international trade may affect hazardous 
exposure in an industry. Perhaps HHE Program physicians routinely participate in 
occupational health professional meetings or electronic dialogues from which 
they receive informal information regarding trends observed in occupational 
medicine clinics and in field investigations. They may then suggest HHE requests 
as a means to a solution. The extent to which such informal channels are used is 
not known. 



BOX 3-1 

Surveillance and the HHE Program 

A strong national occupational health surveillance system could be of benefit to the 
HHE Program, which itself is not a surveillance program. Greater HHE Program rele- 
vance and national impact may be achieved if HHE Program findings were incorporated 
into a national surveillance system in which active searching and reporting of trends and 
clusters are possible. The HHE Program could influence NIOSH management regarding 
the importance of a national surveillance system, perhaps implemented in the Surveil- 
lance Branch of the Division of Respiratory Disease Studies (DRDS). The NIOSH Senti- 
nel Event Notification System for Occupational Risk (SENSOR) program might be used 
to feed data to the system. A good example of an existing system is the reporting of 
asthma by providers to the Massachusetts Occupational Health Surveillance Program 
(Massachusetts Office of Health and Human Services, 2008), Access to comprehensive 
surveillance data could allow the HHE Program to more effective priority-setting among 
HHE requests, make more informed triage decisions, and identify emerging workplace 
health hazards. 



The committee was not provided evidence that the HHE Program conducts 
detailed explorations of workers' compensation or other data to gain insight into 
magnitude and severity of an issue, or to reveal circumstances that could contrib- 
ute to specific hazards. For example, a significant increase in proportionate mor- 
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tality from respiratory disease in cooks was identified in the Washington State 
Department of Health Occupational Mortality Database (Washington State De- 
partment of Heahh, 2008). Such systems may be useful to identify industries or 
workplaces that need investigation, or to identify those needing information re- 
garding the control of hazards. Additionally, outreaeh to workers' compensation 
carriers and loss eontrol consultants may generate new requests for HHEs from 
clients for whieh hazards are identified. 

HHE Program staff members are engaged in professional meetings and na- 
tional conferences, but more could be done to reaeh a greater diversity of employ- 
ers and industries. The HHE Program could use such opportunities to play a more 
active role in identifying potential hazards in emerging industries or proeesses. 
For example, a formal program of seminars, diseussion groups, or other mecha- 
nistie seience routes might enable the HHE Program to link hazardous substanees 
to various industrial uses and identify groups of exposed workers with adverse 
health effects, more than would routine monitoring of national and international 

■J 

databases. Furthermore, the HHE Program could keep traek of ehemicals used in 
the workplace and the potential for hazardous exposure through other sourees. 
Spencer and Schaumburg (2000), for example, list a large number of neurotoxic 
materials. The HHE Program could then actively inform companies, worker ad- 
vocates, and healthcare professionals who may be aware of exposures or apparent 
health effeets in specific populations about the HHE Program and request proeess 
to encourage evaluation of these exposure situations. 

Recommendation: The HHE Program should make systematic use of profes- 
sional meetings, scientific conferences, scientific literature, and surveillance 
data, including those generated by NIOSH, to assist In prioritizing field in- 
vestigations and recognizing emerging issues. 

Finding 4: The extent and effectiveness of relationships between the HHE 
Program and federal and state agencies are variable. 

The committee observed both positive and inadequate interactions of the HHE 
Program with state and federal agencies. 

Through formal and informal reporting relationships with state departments of 
health, labor, unemployment, environment, and workers' compensation programs, 
the HUE Program may help identify worker clusters that could benefit from the 
conduct of HHEs. Trends analyses in unemployment and industry- see tor shifts, 
including identification of pockets of underserved contingent workers, might en- 
hance the HHE Program's ability to reaeh high-risk populations. Maximizing 
HHE Program potential in this area, however, depends on the ability of the pro- 
gram to maintain appropriate staffing levels. These relationships may also allow 



^ Such databases include those of the Environmental Protection Agency, OSHA, the National Cen- 
ter for Environmental Health, the International Agency for Research on Cancer, and the Health 
and Safety Executive in Great Britain. 
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the program to better communicate its mission and activities, and to provide better 
and timelier feedback. 

Three examples demonstrate ways to maximize federal and state resources 
and to enhance the HHE Program's ability to perform its mission. During the 
committee's third meeting (see Appendix C for agenda), a stakeholder from the 
California Department of Public Health spoke with the committee regarding the 
longstanding and strong relationship between California's Occupational Health 
Branch (OHB) and the HHE Program (Matcma, 2008). The relationship has re- 
sulted in capacity building at OHB in a number of issues, including most recently, 
those associated with diacetyl (see Box 3-2 for a discussion about diacetyl). Dur- 
ing the same meeting, management of a worksite where an HHE was conducted 
(NIOSH, 2007J) discussed how useful HHE recommendations were in the even- 
tual elimination of diacetyl in their processes. In this case, three organizations 
worked cooperatively to maximize resources and the result was a positive impact. 



BOX 3-2 
Bronchiolitis obliterans (Popcorn Lung) and Diacetyl 

In 1985, the HHE Program conducted an investigation in an Indiana food processing 
plant where two employees were diagnosed with bronchiolitis obliterans, a severe and 
sometimes fatal lung disease (NIOSH, 1986). Investigators failed to determine the causa- 
tive agent and recommended general industrial hygiene controls. In 1999 and 2000, sev- 
eral current and former workers from a Missouri microwave popcorn plant were diag- 
nosed with bronchiolitis obliterans, and some of these workers were awaiting lung trans- 
plants. Bronchiolitis obliterans is rare in the general population, so the workers' physician 
notified the state health department, which inspected the facility in March of 2000 for risks 
to public — but not worker — health (Michaels et a!., 2008). 

In May 2000, the health department contacted OSHA in Kansas City, described an 
alleged cluster of bronchiolitis obliterans, and requested an inspection for compliance 
with regulations. The request was made of OSHA because it was felt that OSHA could 
"address this situation, and if there is an obvious hazard to workers, address it quickly" 
(Roberts, 2000). The health department did not have the statutory authority to inspect the 
facility without a clear determination of a health hazard. An OSHA inspector visited the 
plant and focused the inspection on well-recognized respiratory hazards: dust and oil 
mist. The Inspector declined to sample for dust because the company's insurance carrier 
had done so earlier. Oil mist samples were collected, but OSHA lab methodologies were 
incompatible for use with vegetable oils. Although OSHA was aware of the bronchiolitis 
obliterans cases, the OSHA area office determined the company to be in compliance and 
closed the file (OSHA, 2000). 

In August 2000, the Missouri health department contacted NIOSH. Over the next 
several months, the HHE Program conducted comprehensive industrial hygiene sampling 
and a health assessment of current employees. The investigation found that rates of ad- 
verse respiratory symptoms were significantly higher than in the general population, and 
that the likely cause was the artificial butter flavoring diacetyl (NIOSH, 2006b). In Decem- 
ber 2000, NIOSH issued interim recommendations to help prevent exposure to harmful 
flavorings (NIOSH, 2000). Over the next three years the HHE Program revisited the plant 
every four to six months to conduct follow-up air sampling and medical testing, and to 
evaluate the effectiveness of the controls it recommended (NIOSH, 2006b). Following its 
initial investigation at the Missouri plant, the HHE Program also conducted evaluations in 
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other locations (Kanwal et al., 2006), issued an alert that provides guidance to the indus- 
try and its workers (NIOSH, 2003e), and generally worked to better understand and 
communicate the risk to workers who produce or apply flavorings, their employers, and 
the occupational health community. 

Diacetyl has generated widespread media attention, perhaps because it is a food 
additive with exposures possible not only among workers who manufacture flavorings, 
but also among consumers. The FDA was alerted in 2007 that a consumer was diag- 
nosed with severe lung disease, whose clinical findings and exposure levels were noted 
to be similar to workers diagnosed with bronchiolitis obliterans (Rose, 2007). An investi- 
gative reporter looked beyond flavoring manufacturing for possible harmful worker expo- 
sure downstream among professional cooks (see, e.g., Schneider, 2008). HHEs are 
pending on diacetyl exposure among professional cooks, but none have been conducted 
to date. The King County, Washington, Health Department has requested factsheets for 
cooks that their restaurant inspectors can distribute during restaurant inspections (Wash- 
ington State Department of Labor & Industries, 2008a). 

In a report from the U.S. House of Representatives Committee on Appropriations 
dated July 2007, concern was expressed about the potential hazards to workers and 
consumers exposed to diacetyl. NIOSH data were cited as compelling evidence of a real 
threat (U.S. Congress, House of Representatives, Committee on Appropriations, 
2007:98). According to that report, the Food and Drug Administration should conduct fur- 
ther studies to examine the safety of diacetyl. In response to public concern and pressure 
from organized labor, OSHA announced in Septemti)er 2007 that it would initiate rulemak- 
ing (Steenhuysen, 2007). In that same month, the U.S. House of Representatives passed 
a bill entitled the Popcorn Workers Lung Disease Prevention Act, directing OSHA to issue 
a standard regulating worker exposure to diacetyl (U.S. Congress, House of Representa- 
tives, House Education and Labor, 2007), currently on the Senate calendar. At the state 
level, legislation was introduced in California to ban the use of diacetyl In the workplace, 
but the bill has been put into an Inactive file of the California Senate (California Assembly, 
2008). As of January 2008, four major popcorn manufacturers voluntarily decided to re- 
move diacetyl from their products (Associated Press, 2007; Schneider, 2007). 

The example of diacetyl and bronchiolitis obliterans, illustrates the HHE Program's 
unique ability to conduct epidemiological investigations and identify previously unknown 
hazards. OSHA and other agencies do not have this same authority or capacity. The pro- 
gram's findings and recommendations have had an immediate effect on the worksites 
investigated, have instigated research in other parts of NIOSH, have influenced voluntar- 
ily changes within industry to remove the hazards, and have had at least some influence 
on policy setting and rulemaking. 



]n another example, an HHE was performed for an employer who managed 
home care services through a Cahfomia county office (10,000 home care workers, 
primarily representing workers from underserved populations) (NIOSH, 2004a). 
Representatives from the county office reported a successful partnering of the 
HHE Program with the Lahor Occupational Health Program (LOHP) of the Uni- 
versity of California-Berkeley (Ayala, 2008). LOHP developed follow-up training 
to assist in implementation of HHE recommendations, which also contributed lo 
the creation of a Labor Management Committee that discussed health and safety, 
as well as other issues. In this example, the HHE Program took advantage of local 
resources to maximize the effectiveness of HHE recommendations. 

A third example of HHE-state occupational health collaboration was provided 
in comments by an industrial hygicnist working at the Massachusetts Department 
of Public Health Occupational Health Surveillance Program (OHSP), which rc- 
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ccivcs NIOSH funding for its occupational asthma surveillance activities {Pcchtcr, 
2008). Over time, three referrals for investigation of potentially hazardous expo- 
sures associated with work-related asthma were identified, and the industrial hy- 
gicnist contacted the HHE Program to conduct investigations at these worksites. 
New-onset asthma associated with exposure to a particular compound {3-amino- 
5-mereapto-triazole) was identified {NIOSH, 2003b; Hnizdo at el., 2004). As 
noted by the stakeholder: 'The identification of a new asthmagen is important, not 
only for the protection of workers currently exposed, but also to the process of 
scientific inquiry about respiratory sensitizers and asthma prevention" {Pechter, 
2008). The HHE Program had previously identified asthma in flock workers at 
one company site {NIOSH, 1998a), while the OHSP request led to identification 
of hazards in a second plant {NIOSH, 2006a; see Box 3-3 for a more detailed de- 
scription of flock and related HHE Program activities). 



BOX 3-3 
Interstitial Pneumonitis (Flock Workers Lung) and Flock 

Flock refers to fine, small-diameter synthetic fibers such as those applied to adhe- 
sive-coated fabrics to produce a pleasing surface similar to velvet. Breathing flock can 
cause a serious lung disease called interstitial pneumonitis which causes inflammation 
and scarring in the lungs. Flock-related illness was first seen in 1991 at a nylon flock 
processing plant in Ontario, although the causative agent was not identified. Investigators 
initially speculated that the condition was caused by mold-contaminated adhesive, which 
was replaced. However, in 1995 the plant reported two additional cases. In 1995 and 
1996, two workers from a Rhode Island plant owned by the same company were diag- 
nosed with interstitial lung disease by the same physician (Lougheed et al.. 1995; Es- 
chenbacher et al., 1999). At the physician's urging, the company asked NIOSH to con- 
duct an HHE. 

The investigation included industrial hygiene surveys of a variety of potential expo- 
sures, respiratory tests, and a medical questionnaire, NIOSH identified flock dust as the 
causative agent and recommended "decisive, proactive action to install effective engi- 
neering controls, to enforce good work practices, to assure appropriate use of proper 
respiratory protection, to establish a medical screening/surveillance program, and to im- 
plement effective administrative controls" (NIOSH, 1996a:33). 

Flock has since been implicated in other outbreaks of occupational lung disease and 
has been the subject of a workshop and recommendations by NIOSH. HHEs have been 
performed In several other flock processing plants. However, flock never generated the 
press attention that diacetyl has, perhaps because it is not a food additive and not a haz- 
ard to consumers. Flock has never been considered for rulemaking by OSHA; the only 
mention of flock on OSHA's website refers to the explosion hazard related to flock dust 
(OSHA, 1998}. One recent follow-up report indicates that, while some industrial hygiene 
controls have been implemented, flock exposures remain high (NIOSH, 2006a). 



Though there are many successes, as described above, stakeholders from both 
local and state public health departments described to the committee the need for 
more information about the HHE Program. The commissioner of health of a 



'' These comments reflect personal opinions and arc not necessarily those ol' ihe Massachusetts 
Department of Public Health. 
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major East Coast cily health department told the committee he knew nothing 
about the HHE Program or of the multiple HHEs condueted in his city over the 
years (Sharfstein, 2007). A Massachusetts OHSP industrial hygienist indicated 
her personal hesitancy in filing HHE requests for other asthma-related issues 
"because, in some eases HHE requests have resulted in long, drawn out processes, 
in which the referring state agency may not be apprised of the process, local 
expertise is not used, the report is not issued for a long time, and the end results 
may not address the most compelling questions in a timely way" (Pechler, 2008). 
These comments suggest the need for both outreach and ongoing communication. 
Better CDC acknowledgement of the depth and breadth of the occupational 
health expertise available to state health departments through the HHE Program 
may result in greater cooperation between the HHE Program and states in identi- 
fying emerging occupational health issues. For example, the recent identification 
of a new form of progressive inflammatory neuropathy among workers processing 
pig brains (MMWR, 2008) might have benefitted from earher attention to the 
work-relatedness of the initial complaints. A number of prior CDC investigations, 
including the initial evaluation of a case of pulmonary anthrax in Florida 
(MMWR, 2001a), suggest a pattern of delay in including appropriate occupational 
health expertise in investigations. The HHE Program could more broadly influ- 
ence workplace health assessments conducted by other CDC institutes by working 
more closely with CDC overall.^ Additionally, the HHE Program's identity with 
the public may be diminished as the 1-800-66-NIOSH telephone number formerly 
used for contacting NIOSH about occupational safety and health issues is sub- 
sumed under the general CDC call line. NIOSH could maintain visibility with 
other agencies and the public if it were able to reinstate that telephone number. 

Recommendation: Enhance HHE Program outreach to OSHA national and 
regional offices and to state health and labor departments to better commu- 
nicate the function and activities of the HHE Program, increase cooperation 
with these agencies, and provide more complete and timely feedback. 



RELEVANCE OF THE HEALTH HAZARD EVALUATION PROGRAM 

ACTIVITIES 

Finding 5: The HHE Program has responded well to HHE requests as 
mandated, although mechanisms for eliciting a broader array of HHE 
requests are needed. 

There is strong evidence the HHE Program has conducted relevant field inves- 
tigations and has performed well in identifying emerging hazards, serious occupa- 
tional health problems, and hazards affecting large numbers of workers. Examples 
of such cases include (I) bronchiolitis obliterans (popcorn lung) associated with 



■ HUE Program staff indicaled that currently available communication pathways within the CDC 
(such as Epi-X) may lack limellness or completeness (NIOSH. 2007d, 2008b). 
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diacetyl in the manufacture of flavorings, (2) interstitial pneumonitis and flock 
exposures; (3) work-related musculoskeletal disorders associated with cumulative 
trauma, and (4) biological and chemical hazards in the meat and poultry indus- 
tries. These examples are described more fully in Boxes 3-2, 3-3, 3-4, and 3-5, 
respectively. All four examples included multiple field investigations and success- 
ful transfer of knowledge to the occupational health community by the HHE Pro- 
gram. 



"Casting a Larger Net" 

Based on information submitted to the committee, it is not possible to deter- 
mine whether the needs of underserved populations and small businesses are be- 
ing met or whether the lack of anonymity or the fear of legal or political reprisals 
inhibits the request of HHEs. Targeted outreach to state health, labor, and envi- 
ronmental departments; small business and their employees; labor unions; health 
centers providing eare to underserved populations; community and ethnic organi- 
zations; and nongovernmental worker centers serving immigrant and contingent 
workers would increase the likelihood of meeting the needs of these populations. 
It is especially important to produce educational materials in multiple languages 
appropriate to these diverse populations. HHE Program outreach would benefit 
from provisions for assisting requestors, particularly those who are not unionized 
or who belong to underserved populations, in formulating valid requests. 

Communication with non-unionized and underserved populations could be 
enhanced through collaboration with emerging worker centers that address the 
needs of new immigrant workers; public service announcements geared toward 
truck drivers, agricultural, and construction workers on radio stations and website 
links (for example, on trucker websites); or targeted outreach to health centers 
providing eare to underserved populations. 

The BSC recommendation to "east a larger net" to enable the program to "se- 
lect only the evaluations that truly serve program goals" (NIOSH BSC, 2006:5) 
deserves attention here: while many attempts have been described to increase the 
overall number of HHE requests, there has been no systematic effort to ascertain 
either the relative success of these efforts or the causes of any failures. Specific 
approaches to identify and encourage new sources of appropriate and valid re- 
quests do not appear to have been evaluated. It would be useful if outreach pro- 
grams could be evaluated for effectiveness. 

Ongoing systematic scanning of surveillance data could assist the HHE Pro- 
gram in identifying emerging hazards in underserved populations. While there are 
no specific plans to identify and control hazards that speeifieally affect under- 
served populations, it is possible to focus resources in industries and workplaces 
that employ a significant proportion of underserved workers. 

Recommendation: Establish formal relationships with organizations repre- 
senting underserved populations, small businesses, and their employees. 
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Recommendation: Use innovative techniques to reach small businesses and 
underserved populations, creating a broad array of mechanisms for commu- 
nicating with diverse constituencies and attending to issues of literacy, lan- 
guage, and national-origin barriers. The effectiveness of applied outreach 
should be evaluated in a formal manner. 



The Health Hazard Evaluation Program Triage Process 

Finding 6: The HHE triage process is generally efficient but requires 
more structure. 

Formalization of the Triage and Response Processes 

Program evaluations by the Research Triangle Institute (RTI) and the BSC 
emphasized the need for the HHE Program to conserve searce resources when ad- 
dressing its core mission (RTI, 1997; NIOSH BSC, 2006). In response to BSC 
recommendations, the HHE Program introduced the triage system described in 
Chapter 2 (NIOSH BSC, 1997). The HHE Program developed structured triage 
mechanisms that allows the delivery of standard information packages to request- 
ors when full investigations were not warranted (for example, requests triaged as 
category 1 or 2; see Box 2-2). Many lEQ-related requests may fall under this 
category. 

The criteria applied during triage, apart from the straightforward assessment 
of request validity as delennined by regulation, do not explicitly outline when 
field investigations are necessary, nor do the criteria assist in distinguishing those 
requests that require complex investigations or that may yield fiiture research op- 
portunities. Classification appears to be handled informally. Initial determinations 
are not always final; project officers can change the category of response to a re- 
quest after additional information is obtained. 

A clear understanding of an HHE request and the implications of a potential 
response should be explicit in any triage decision. For example, a letter response 
to a requestor reviewed by the committee documented an attempt by HHE Pro- 
gram staff to provide thoughtful review and useful reference information, but in- 
cludes conclusions based on limited information when resources may have been 
available elsewhere to address the concerns in greater depth. The report involved 
a request from management of a muki-nalional petroleum company to explore a 
possible cluster of esophageal cancer (NIOSH, 2005c). The HHE Program was 
asked ''to evaluate the strengths, weaknesses, and validity of [a prior industry- 
conducted study at the worksite] and to give suggestions or recommendations for 
further investigation." A thorough in-housc response to this request would have 
consumed excessive HHE Program resources. The HHE letter report provided a 
general explanation about the limitations of cancer cluster investigations, and rec- 
ommended no further investigation of the suspected cancer cluster. 
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BOX 3^ 
Work-Related Musculoskeletal Disorders 

Work-related musculoskeletal disorders (WMSDs) are disorders of the muscles, ten- 
dons, joints, and associated neurovascular structures that occur over time rather than in 
response to an acute traumatic event. Examples include rotator cuff syndrome at the 
shoulder, lateral epicondylitis at the elbow (tennis elbow), carpal tunnel syndrome at the 
wrist, sciatica, and back pain. While these disorders can be unrelated to work activities, 
their work-relatedness is associated with repetitive work, awkward postures, and overex- 
ertion in lifting, pushing, pulling, or carrying, as well as hand-arm vibration. The likelihood 
of WMSDs increases with the intensity, duration, or frequency of exposure to these risk 
factors or when multiple risk factors occur simultaneously. These disorders represent 30 
to 40 percent of all reported occupational injuries and illnesses, lost work days, and 
workers' compensation costs. In an example from Washington State, a 26-year-old fe- 
male nursing assistant employed in an adult family home for seven years developed back 
problems lifting and transferring residents. She missed 180 days of work and required 
$5,200 of medical treatment {Silversteln and Adams, 2007). 

The HHE Program has played a pivotal role in the area of WMSDs, demonstrating 
very high relevance for their HHE findings related to hazard identification and abatement. 
Eighty-two WMSD-related HHEs were conducted between 1980 and 2007 {19 of them 
between 2000 and 2007). Early HHEs in food processing and other manufacturing 
brought public, research, and regulatory attention to the area. Experiences gained in the 
HHE Program resulted in a series of peer-reviewed and practice-oriented publications, 
including a comprehensive review of the literature on WMSDs (NIOSH, 1997b). The 
comprehensive review was referenced in the European Union (Buckle and Devereau, 
1999), was used extensively in the development of regulations at the national level, and 
was used to justify the making of ergonomic rules in at least one state^ (Washington 
State Department of Labor & Industries, 2000). Many of these HHEs were also used in 
various rulemaking efforts for both evidence of hazards and viable solutions (Washington 
State Department of Labor & Industries, 2008b.), demonstrating a high degree of regula- 
tory relevance. A later congressionally requested review by the National Academies sub- 
stantiated the earlier HHE Program work (NRC and lOM, 2001). Other documents based 
on HHE Program outputs include several manuals to assist employers and employees in 
the development of effective ergonomics programs, including those intended for under- 
served populations such as farm workers (NIOSH, 1997a, 2001e, 2007k), These docu- 
ments have been widely circulated within the health and safety practitioner community, 
as well as to small and large employer and worker organizations at professional confer- 
ences. A number of these ergonomics publications have been adapted and distributed by 
the HHE Program. 

^Washington Stale adopted an ergonomics rule in May 2000 (WAC 296-62-05101) 
{http://www.lni.wa.g0v/WISHA/Rules/GeneralOccupati0nalHealth/PDFs/Erg0RulewithAppendices.p 
df . accessed August 9, 2008). 
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BOX 3-5 
Biological and Chemical Hazards in the Meat and Poultry Industries 

The HHE Program has identified and addressed several emerging hazards and con- 
tinues to play a major role in investigating risks in the expanding meat and poultry proc- 
essing industry, an industry increasingly comprised of a non-union, minority, immigrant, 
and contingent workforce characterized by low wages and high risk (GAO, 2005b). The 
HHE Program, working with the State of North Carolina and the U.S. Department of Agri- 
culture (USDA), identified the occupational transmission of brucellosis to workers in 
swine processing facilities (NIOSH, 2GG7b). Brucellosis is an infectious bacterial disease 
that can cause "a range of symptoms that are similar to the flu and may include fever, 
sweats, headaches, back pains, and physical weakness. Severe infections of the central 
nervous systems or lining of the heart may occur. Brucellosis can also cause long-lasting 
or chronic symptoms that include recurrent fevers, joint pain, and fatigue" (NIOSH, 
2007b:57) An HHE investigation (NIOSH, 1994) provided the infomnation needed for the 
USDA to develop and implement a formal rule change that provides compensation for 
swine herds destroyed because of infection, effectively eliminating the hazard by remov- 
ing disease-carrying animals from the food chain. This outcome reflects an extraordinary 
level of cooperation among both federal and slate agencies. 

In another case, the HHE Program conducted initial and full follow-up investigations 
of a management-initiated request to evaluate respiratory complaints in a poultry proc- 
essing area in which hyperchlohnated water was used in the evisceration process 
(NIOSH 2003a. 2006f). The report related to an initial HHE investigation demonstrated 
excess levels of chlorine and thchloramines generated by this process and the subse- 
quent reduction of exposure following elimination of hyperchlorinated water. The final re- 
port clearly documents the impact of an aggressive workplace management approach 
that went beyond the initial HHE Program recommendations and effectively reduced the 
hazard and improved outcomes in that plant. During the Initial investigation, NIOSH engi- 
neers encouraged plant management and engineers to take the Issue seriously. How- 
ever, there is no indication these recommendations have been applied more broadly in 
the poultry processing industry. 

Reported more recently in the news is an outbreak of progressive inflammatory neu- 
ropathy among swine slaughterhouse workers (MMWR, 2008; CNN.com/health, 2008). The 
CDC was called to investigate this as a potential contagious disease. The inclusion of the 
HHE Program in the Investigation has facilitated the broader evaluation of the condition as 
an occupational illness and further demonstrates the relevance of the HHE Program. 



Conveying this type of information effeetively via telephone and written eon- 
tact poses a diffieult risk-eommunication challenge, but this eommunication 
means may be usctiil when scarce HHE resources are unlikely to provide useful 
new information for workers. It eould be perceived, however, that the HHE Pro- 
gram was used to validate the company's research without sufficient critical per- 
spective brought to bear. If neutral third-parly expertise is required, large entities 
with adequate resources eould be encouraged to develop tripartite oversight (by 
NIOSH, labor, and industry) of external experts and formally sponsor a thorough 
review. If the information is potentially of major interest, this could be triagcd to 
other areas of NIOSH. 
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It is unclear how many letter reports generated by the HHE Program would 
raise similar issues. The decision against using searee resources to conduct full 
site investigations (for category 3 or 4 requests) should not be used to provide ap- 
proval or premature conclusions. A description of the needed resources — 
including, when appropriate, independent extramural research expertise — and al- 
ternative approaches to addressing the issue would also be useful. The HHE Pro- 
gram would benefit from a formal mechanism to respond by recommending tri- 
partite studies, funded by the requesting entity when appropriate. General guide- 
lines should exist for referral to higher-level review, particularly for requests in- 
volving large companies with multiple resources, or for requests that would resuU 
in the provision of recommendations for which HHE staff may not be sufficiently 
expert or have the appropriate certifications to make. When sufficient information 
about the hazard or issue is readily available, requestors might be referred to the 
OSHA Small Business Consultation Program or appropriate OSHA directorate 
(for example, the Directorate of Cooperative and State Programs, or the Director- 
ate of Science Technology and Medicine). 



Identification of Needed Expertise 

A review of several HHE letter reports by the committee indicates that the tri- 
age process may not be effective in determining whether staff expertise is suitable 
for the range of potential program responses to a request. The triage program 
would ideally involve epidemiologists, toxicologists, engineers, and other relevant 
specialists, as well as HHE Program staff, to ensure that qualified personnel arc 
handling responses. For example, the committee reviewed a close-out memo from 
an HHE Program file regarding a state health department request for assistance 
related to possible worksite metals contamination (NfOSH, 2007a). The memo 
documented a medical review by HHE Program staff and the provision of diag- 
nostic information by telephone, although the recipient of the information was the 
public health official who requested the assistance, and not a clinician responsible 
for the individual with the health concerns. HHE Program personnel, of necessity, 
are generalists. Slightly more than half of HHE personnel have basic certification, 
such as certified industrial hygienist or medical board certification in a primary 
specialty, but none have additional toxieological certification (NiOSH, 2007b). 
Further, the provision of clinical advice requires a therapeutic relationship with a 
patient. The memo does not state whether the HHE Program provided the indi- 
vidual's treating physician with contact information of an appropriately qualified 
medical toxicologist for a clinical consultation. ' 



^ The American Board of Medical Speciallies detlnes medical toxicology as the specialization in 
the "prevention, evaluation, treatment and monitoring of injury and illness from exposures to 
drugs and chemicals, as well as biological and radiological agents" 

(http://www.abms.org/Who_We_Help/Consumers/About_Physician Specialties/preventive. aspx, 
accessed September 18, 2008), Three years of specialized training is required to receive a certifi- 
cate in the subspecialty of preventative medicine. 
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Inclusion of specialists in the triage process would also help identify requests 
where exposure responses are potentially unusual or not readily explicable on the 
basis of current knowledge, potentially leading to a higher categorization of the 
HHE request. 



Transparency of the Process 

The committee reviewed input from numerous program stakeholders. In gen- 
eral, HHE Program guidance and recommendations were considered well rea- 
soned and sound, but some comments from stakeholders raised issues related to 
the triage process. Multiple stakeholders indicated not understanding why their 
HHE requests were denied. The committee does not have enough information to 
determine whether this eoneem is widespread, but did determine that greater 
transparency in the triage process would, at the very least, help the program de- 
termine whether the confusion is avoidable. Responding to HHE requests with a 
systematic approach of defining the request, identifying the impHeations of poten- 
tial responses, and identifying justifiable resources and necessary expertise based 
on the triage criteria would provide the means for the program to better communi- 
cate the outcomes of the triage process with requestors. A more transparent triage 
process could potentially help requestors understand why their requests may be 
denied and help them form valid requests in the fiilure. 

Recommendation: Implement, as part of the triage process, a formal techni- 
cal assistance mechanism to help requestors to formulate valid HHE re- 
quests. In cases where an HHE is not appropriate or where resource limita- 
tions prohibit an investigation, technical assistance should include referral to 
more appropriate NIOSH divisions or government agencies. 

Recommendation: Develop an explicit, written process for classifying and 
prioritizing HHE requests. Priority should be based on the gravity of the po- 
tential harm, the number of employees potentially at risk at similar work- 
places or using similar work processes, the urgency of the problem, the po- 
tential to assess health outcomes, and the possibility of identifying emerging 
issues. Potential impact on standards and policy should also enhance the pri- 
ority of an HHE request in the triage process. Relationship of the HHE to 
current research may be considered, but should not be the only or primary 
factor. The process should provide guidance on weighting these varying fac- 
tors. 

Recommendation: Better formalize the triage process, including the identifi- 
cation of needed expertise, and improve the transparency of the process to 
HHE requestors, while maintaining flexibility and speed. 
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RELEVANCE OF OUTPUTS 

Finding 7: HHE reports are generally well written, present relevant in- 
formation supported by appropriate documentation, and reflect a high 
level of expertise. However, the committee did not find evidence that a 
well-defined quality assurance process is in place to ensure consistently 
high-quality outputs and recommendations. 



Review of Numbered Reports 

The HHE Program provided the committee with several examples of num- 
bered HHE reports as well as less formal letter responses to HHE requests on a 
variety of occupational health-related issues. The committee also reviewed several 
other numbered HHE reports obtained directly from the HHE Program website. 
The majority of listed authors of numbered HHE reports reviewed held advanced 
degrees and certifications, which enhances credibility and professionalism. The 
report formats were uniform and straightforward with specific recommendations 
provided in clear language at the outset, followed by a plain-language summary, 
then the body of the report. All HHE field investigation reports were dated, and 
nearly all include both the date of initial request as well as the source of the re- 
quest (for example, management, employees, union). 

Report timeliness, a major concern expressed in all previous HHE Program 
evaluations, continues to vary, with the most rapidly delivered reports delivered 
four to six months from the request date, and the majority appearing within a year 
of the field investigation. One outlier, the final report for an incident that recog- 
nized the association between flavorings and lung disease (NIOSH, 2006b), ap- 
peared five-and-a-half years after the initial request and two-and-a-half years after 
the final follow-up visit. Reasons for the delay were not apparent to the commit- 
tee. Timeliness, however, was an issue only for the final numbered report. This 
particular investigation was highly relevant and productive. The hazard was iden- 
tified and reported to workers and management in a series of timely letter reports. 
The investigation resulted in new research activities, peer-reviewed publications, 
and policy efforts. The final report included all results in tabular form, and an ex- 
haustive set of appendixes that captured relevant correspondence and other mate- 
rials, comprising nearly 200 pages of clear mformation. Many other reports deliv- 
ered more quickly also contained clear information, tables of exposure and out- 
come data, and relevant references. 

Public accessibility to more complete data is extremely important for program 
relevance and impact. It would be useful to determine whether the traditional 
HHE numbered report is the appropriate venue to provide this access, given the 
time taken to produce some of these reports, or whether more rapid access to ap- 
propriately vetted data and reports might provide a more effective and efficient 
alternafive. Providing comprehensive, accessible data is now a requirement for 
large-scale National Institutes of Health-sponsored extramural research, and the 
online provision of questionnaires, methods descriptions, and workplace diagrams 
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allows accessibility of a more complete data set than that available through peer- 
reviewed published sources. Peer-reviewed publication, however, is also a criti- 
cally important feature of all scientific discovery including HHEs, and the HHE 
Program has been successful in producing such outcomes as a result of some of its 
investigations. The HHE Program could potentially increase the relevance (and 
impact) of its activities by determining whether data and information collected, or 
recommendations made during the course of an HHE, should be publicly released 
prior to the publication of its numbered reports. 

The numbered and letter reports in the committee's sample demonstrate the 
depth of expertise available to the HHH Program. Expertise from other parts of 
NIOSH was included in the investigation of hazards such as noise and electro- 
magnetic field exposures. A high level of interagency cooperation and interaction 
with a variety of stakeholders in developing outreach materials was displayed, and 
these interactions were often relevant for policy development within NIOSH and 
beyond. The quality and depth of interaction between HHE Program staff and 
stakeholders is evident in major examples described in Boxes 3-2 through 3-5, as 
well as for information developed by the HHE Program concerning latex expo- 
sure (NIOSH, 1997c, 1998d), publications related to carbon monoxide as a preva- 
lent and lethal hazard in outdoor settings (NIOSH, 1996b; U.S. Coast Guard, 
2001, 2008; U.S. National Park Service, 2005); and the enhancement of under- 
standing of indoor air quality issues (EPA and NIOSH, i998; Mendell et al., 
2002; Kreiss, 2005), among other topics. 



Review of Letter Reports 

The HHE Program provided the committee with 10 letter reports written in 
response to fiill site visit investigations, and 5 examples of letter and fax re- 
sponses provided when site visits were considered unwarranted. The letters in- 
cluded names and contact information of the program personnel responsible for 
generating the report, and telephone contact was freely offered. Two of the letter 
responses were apparently written using a template, indicating efficient use of re- 
sources. Each was appropriately adapted to the specific concerns of the complain- 
ant and included useful references to bolster the generalized response. 

All the letter reports could benefit from a short list of recommendations in 
priority order, similar to what is provided in numbered reports, so that clear and 
immediate information may be provided. Apart from this distinction, the letter 
reports reviewed were generally of as high a quality as the numbered reports, and 
often included appropriate references and extensive reports of findings in tables 
and appendixes. The examples provided reflect a response to customer service 
needs for direct contact and timeliness. Seven of the 10 site visit letter responses 
indicated the date of request. The time from request to final response ranged from 
I to 1 1 months, including 4 letters issued 5 or more months after the request. 

One letter report appeared extremely helpful because it included contact in- 
formation for local and state personnel who could address specific requestor con- 
cerns. In another report, a nonspecific recommendation for medical evaluation for 
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personnel with indoor air quality complaints (NIOSH, 2007a) would have been 
more helpful if accompanied by a link to professional resources such as the Asso- 
ciation of Occupational and Environmental Clinics. The letter reports related to 
lEQ issues (NIOSH, 2007e, for example) described but did not provide reference 
to an Institute of Medicine (lOM) report. A website link to the lOM report could 
have been provided. 



Stakeholder Satisfaction and the Followback Program 

To determine the level of satisfaction with HHE field investigations and re- 
ports, the committee reviewed the responses to followback questionnaires 
(NIOSH, 2007b, f). The HHE process and reports were generally highly valued 
and useful, and some recommendations were being implemented. Timeliness of 
reporting still needed improvement, and stakeholders sometimes raised concerns 
regarding the feasibility or technical accuracy of HHE recommendations. Some 
stakeholders said they did not understand why their HHE requests were not 
deemed appropriate for field investigations. A high percentage of followback 
questionnaire respondents indicated that HHE Program staff members were very 
professional and knowledgeable, and that the HHEs were conducted in a profes- 
sional manner. Nearly all stakeholders who provided information to this commit- 
tee were impressed with the level and breadth of expertise demonstrated by HHE 
staff. Based on reports reviewed by this committee, the HHE Program staff mem- 
bers are well aware of the literature regarding different hazards or potential haz- 
ards associated with the requests they are receiving. 

Improvements could be made to avoid selection bias in followback program 
results by identifying similar facilities for which HHEs were not requested, and by 
comparing health hazards before and after investigations. Follow-up health ques- 
tionnaires, examinations, or exposure monitoring could show significant improve- 
ment. These exercises, however, require resources the HHE Program may not have. 
Nonetheless, the committee believes the followback program is an important tool 
for program process improvement, and could be made more valuable with input 
from program evaluation expertise. Current strategic goals include performance 
measures that would expand the current followback questionnaire feedback loop to 
address program impact and improve customer satisfaction. Continued efforts to 
improve the timeliness of reports, improve written and telephone eommunication, 
and enhance dissemination efforts were listed by the HHE Program as measurable 
objectives to improve customer response (see Table 3-1). No information was re- 
layed regarding the resources required to accomplish the objeelives. 



Quality of Recommendations 

The committee assessed the quality of recommendations contained in HHE 
reports, and many were excellent. As one example, in response to an employee- 
generated request for an HHE, the HHE Program conducted an evaluation of a 
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government- owned, eon trae tor-opera ted hazardous waste "tank farm." A great 
deal of negotiation with workplaee management was required to eonduet this in- 
vestigation, but the result was a rapidly produeed, thoughtful, and thoroughly ref- 
ereneed report that provided an overarehing reeommendation embedded with very 
speeifie reeommcndations (NIOSH, 2004b). This approaeh ereated a framework 
for managing speeifie problems across a complex worksite with multiple layers of 
responsibility. It provided a model that synthesized both the problems and the ap- 
proaches to solving them. 

The committee found several reports and recommendations that do not exhibit 
the same level of quality. For example, one reviewed report that was focused on 
musculoskeletal issues and recommendations. A variety of other potential safety 
hazards were identified, such as inadequately plumbed eyewashes and slippery 
surfaces, but only superficial recommendations to control these hazards were pro- 
vided that did not include references for more detailed information (NIOSH, 
2005a). Another report, issued 18 months after the initial request, clearly de- 
scribed serious outdoor carbon monoxide exposures, but offered only generic rec- 
ommendations for remediation despite the evidence of ongoing work by other sci- 
entists in NIOSH to develop engineering controls (NIOSH, 2004e). 

The quality and utility of the HHE Program recommendations have been tar- 
geted for improvement during previous program reviews (NIOSH BSC, 1997, 
2006; RTI, 1997). A systematic response on the part of the HHE Program to en- 
hance the quality of recommendations does not appear to have been made. The 
mixed quality of the recommendations in the reports and letters reviewed by this 
committee suggest that quality control measures could be upgraded. At present, 
technical and policy review is conducted at the branch level within NIOSH: only 
rarely, and at the discretion of the branch chief, is other scientific quality review 
obtained. Given the understandable tension between timeliness and the inevitable 
delays additional layers of review would entail, the committee recommends a 
sampling strategy of recent reports for review for scientific content and especially 
for accuracy of recommendations. The strategy would be used primarily as a qual- 
ity improvement or training mechanism. Such reviews could be obtained from 
scientists elsewhere in NlOSll, from extramural scientists, from practitioners in 
labor and industry, and perhaps from OSHA or the Mine Safety and Health Ad- 
ministration (MSHA). Toxieologists, epidemiologists, engineers, and others from 
academic research institutions or from prioritized industry sectors might review 
both for quality and to identify emerging issues. 

Recommendation: Ensure that recommendations in HHEs are relevant, fea- 
sible, effective, and clearly explained. Such steps may include: 

a. Explanation of the relevance, feasibility, and impact of each recom- 
mendation in the text of HHE Reports. 

b. Priority-setting among recommendations in all reports to indicate 
those requiring immediate action in the targeted workplace. 

c. Debriefing in NIOSH after site visits and report dissemination for de- 
termination of relevance and impact on a systematic basis (potentially 
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missed opportunities to identify emerging health hazards could also 
be Identified). 

d. Modification of the followback surveys for use in assessing the rele- 
vance, feasibility, and impact of recommendations. 

e. Enhancement of internal quality assurance by development of a for- 
mal program and consideration of sending a sampling of recent re- 
ports and technical assistance letters for external review of scientific 
content, report completeness, and appropriateness of recommenda- 
tions. 



EMERGING ISSUES 

Finding 8: The HHE Program is an effective tool for uncovering emerg- 
ing issues. 

Emerging issues in occupational health include newly created hazards such as 
those arising from nanotechnology;^ newly discovered hazards such as diacetyl 
flavoring; and known hazards in new or previously overlooked populations, such 
as silica exposure among roofers. Nanoparticles^ are recognized as a potential 
health hazard and have been identified as an emerging issue by the National 
Academies Committee to Review the NIOSH Mining Safety and Health Research 
Program (NRC and lOM, 2007). As described in Chapter 2. OSHA has the lead in 
controlling well-understood hazards through legally enforceable standards. 
NIOSH has the responsibility to investigate new and emerging hazards for which 
standards do not exist or may be inadequate. 

The HHE Program is an important mechanism within NIOSH for identifying 
and investigating emerging issues. The results of HHEs are similar to case reports 
in the medical literature; they arc not always as definitive or as easily generalized 
to olher workplaces as are the results of epidemiological research. But the HHE 
Program can take action on emerging issues in much less time, for much less 
money, and with much more flexibility to modify investigations in response to 
changing circumstances. One of the difficulties NIOSH and the HHE Program 
face in adapting the program to address emerging issues is that HHEs arc done in 
response to requests, rather than being self-initiated. Nevertheless, NIOSH and 
the HHE Program could do more lo track emerging i.ssucs and promote appropri- 
ate requests to address emerging issues. 



Nanotechnology, sometimes called mo/cciilar manu/aclun'ng, is a branch of engineering that 
deals with the design and manufactnre of extremely small electronic circuits and mechanical de- 
vices built at the molecular scale. Nanoiechnology is often discussed together with microeleclro- 
mechanical systems (MEMS), a subject that usually inclndes nanotechnology but may also include 
technologies higher than the molecular level 

(http://whatis.techtarget,coni/deflnition/0„sid9_gci213444,00.html, accessed March 28. 2008). 
** A nanopartii'le is a microscopic particle with an aerodynamic diameter of less than 100 nm. They 
are sometimes referred to as ultrafmc particles 
(http://sis.n!m.nih,go\7enviro/iupacglossary/glossaryn.html, accessed August 6, 2008). 
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A resource for stimulating HHE requests related lo emerging hazards is the 26 
state OSHA programs, all of which have enforcement and consultation resources, 
and all of which are members of the Occupational Safety and Health State Plan 
Association. To date, however, there is no evidence of interaction with this or- 
ganization. The program could establish a stakeholder group or groups to assist in 
identifying exposure circumstances or types of workplaces that could be the ob- 
ject of HHE requests likely to have high relevance or impact. The NORA sector 
councils may serve this function. 



Newly Identified Hazards 

Work Organization hsiies 

Work organization describes how work is managed and organized. Issues in- 
clude long or irregular hours, awkward or repetitive motion, job changes due to 
new technologies, and excessive or conflicting job demands. Twenty years ago, 
work organization issues were all but ignored; today, there is a growing recogni- 
tion that work organization can have a major impact on worker health. Two of the 
most common problems arc musculoskeletal disorders and stress-related condi- 
tions. Work organization issues can arise in any working population, but arc espe- 
cially important in healthcare and other service industries, and in disaster response. 

A number of HHEs have addressed work organization and its impact on 
health. They include HHEs of musculoskeletal disorders at the Los Angeles Times 
(NIOSH, 1990); of electronic headset noise exposure among transcribers at Kai- 
ser-Permanent c in California (NIOSH, 2005b); of respiratory, gastrointestinal, 
dermal and stress-related disorders among members of the New Orleans Police 
Department after Hurricane Katrina (NIOSH, 2006d); and of respiratory disease 
and mental disorders among New York transit workers exposed in the 9/1 1 attack 
on the World Trade Center (NIOSH, 2005c). NIOSH has recognized that work 
organization issues are doubly relevant to its mission: even if they themselves do 
not cause death and disability, they may increase an afflicted worker's vulnerabil- 
ity to more traditional health and safety hazards. The HHE Program is instrumen- 
tal in identifying and responding to emerging issues associated with work organi- 
zation. 



Chemical Hazards 

To be more proactive, the HHE Program and NIOSH might consider certain 
classes of chemical compounds as emerging issues. For example, organic alde- 
hydes such as acrolein are highly reactive compounds and widely used in certain 
industrial processes. Acrolein, used in the production of acrylic acid, has known 
toxic properties and may be carcinogenic (ATSDR, 2007). Quinones arc another 
group of chemicals of potential concern. Occupational exposure to quinones may 
occur in the dye, textile, chemical, tanning, photographic processing, and cos- 
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metic industries (OSHA, 2007). NIOSH and the HHE Program could determine 
the industries in which these and other potentially hazardous compounds are used, 
and determine whether existing permissible exposure limits (PELs) are adequate. 



New Technologies 

Few NIOSH HHEs have dealt with the hazards of major new technologies, 
and much more could be done in this area. For example, NIOSH has published 
only one HHE in the area of nanotechnology- — an investigation of carbon fiber 
exposure in a University of Dayton research lab (NIOSH, 2006i). That investiga- 
tion did not include medical evaluations and focused on traditional industrial hy- 
giene controls for fine particulates. However, an increasing number of workers 
are exposed to nanomaterials, not only in laboratories, but in the manufacturing of 
products such as cosmetics. In the future, NIOSH will be challenged by a rapidly 
increasing use of nanotechnology in the workplace, and by other new technolo- 
gies such as biologically engineered products and manufacturing methods. The 
HHE Program should be able to make use of the research in the respiratory pro- 
gram at NIOSH to address these issues. 

Other emerging issues could result from the use of genetic engineering in the 
pharmaceutical industry that may generate unforeseen chemical and biological 
exposures. The HHE Program could identify these issues through NORA sectors, 
improved links with local and state heahh departments, and with surveillance data 
(see below). Emerging issues could also be idcnfified through rapid response to 
requests in this area and identifying where these products may be generated and 
contacting employers and employee organizations about HHE requests. 



Known Hazards Affecting Underserved Populations and Small Businesses 

It is an HHE Program priority to honor HHE requests. However, if the pro- 
gram is to be fully relevant, it should seek out emerging issues by ensuring that 
workers and employers know about the HHE Program and understand the value 
of filing a request. For the most part, this has not been done. The program has 
passively relied on requests from a variety of sources rather than actively seeking 
out worksites with new and emerging hazards. Such worksites could be identified 
through the occupational health literature, including international publications; 
engineering and trade association literature on new technology; databases of in- 
dustrial chemicals and their uses; and the systematic use of state occupational 
health surveillance systems. A good first step might be the development of a sys- 
tematic approach to facilitate identification of known hazards where PELs may be 
inadequate or nonexistent, and identification of the workplaces where the hazards 
may be encountered. A rich source of ideas may be NIOSH 's own files. It would 
be especially useful for NIOSH and the HHE Program to maintain and systemafi- 
cally review a tickler file of odd and unexplained findings to be reviewed periodi- 
cally for follow-up and further investigation. 
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Previous sections of this chapter have addressed the need for the HHE Pro- 
gram to better eommunicate with non-union workers and underserved popula- 
tions. Emerging issues arc particularly difficult to identify among these workers, 
specifically because of the lack of communication with them. Novel means of 
outreach to these populations might include community organization and worker 
centers, as well as publicly flinded health clinics. 

Recommendation: Initiate formal periodic assessment of new and emerging 
hazards. The committee recommends the following steps: 

a. Evolve from a program that passively receives requests to a proactive 
program that seeks opportunities for field investigations. 

b. Develop systematic approaches to identify hazards where OSHA 
PELS are inadequate or nonexistent, to identify unknown hazards, 
and to identify known hazards encountered under new circumstances. 

c. Establish and periodically review a tickler file of inconclusive or un- 
expected evaluation results to determine whether new trends or prob- 
lems may be emerging. 

d. Periodically meet with intramural and extramural research scientists 
and stakeholders in government, academe, labor, and industry to dis- 
cuss specific unresolved evaluations, to review aggregate findings, and 
to solicit input about new or emerging hazards or interventions. The 
HHE Program could establish one or more stakeholder groups to as- 
sist in identifying exposure circumstances or types of workplaces that 
could be the object of HHE requests likely to have high relevance and 
impact. The NORA sector councils may serve this function. 



EMERGENCY RESPONSE 

Finding 9: The HHE Program is uniquely qualified to formulate informa- 
tion needed to safeguard the workforce responding during and following 
a disaster. The program is well prepared to deploy during emergencies. 

Emergency preparedness is one of the most relevant aspects of the HHE Pro- 
gram. The strength of the HHE Program in emergency response comes from the 
vast experience staff gain through day-to-day activities, particularly through en- 
gaging with employers, workers, and unions at workplaces throughout the coun- 
try. Essential abilities are developed, such as improvisation and flexibility, which 
lend themselves to emergency response. As described in Chapter 2, the HHE Pro- 
gram has served an important role in the response to natural and manmade disas- 
ters. HHE personnel were essential to NlOSH's response to Hurricanes Katrina 
and Rita, largely because of their expertise in staging investigations under adverse 
conditions and writing recommendations in clear language without technical jar- 
gon. The NIOSH response was rapid, efficient, and competent. Within days, 
NIOSll produced and disseminated onsite guidance for emergency responders. 
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medical personnel, and elean-up workers. According to a retired employee of the 
U.S. General Services Administration, Office of Govemmentwide Policy, Office 
of Mail Communications, the HHE Program had a role that was both relevant and 
appropriate during the anthrax crisis, helping to ensure the safety and security of 
federal employees (Bender, 2008). 

Since 9/11, the program has reasonably and appropriately focused on the all- 
hazards scenario — chemical, biological, radiological, nuclear, and high yield ex- 
plosives events — working with other NIOSH programs responsible for informa- 
fion. The HHE Program works with OSHA in a joint operations capacity: the 
HHE Program provides advice and guidance to OSHA on health hazards and is- 
sues, while OSHA focuses primarily on safety. 



Review of Outputs Related to Emergency Response 

The committee reviewed two numbered reports related to emergency response 
submitted by the HHE Program (NIOSH, 2004d, 2006d). Both were considered 
extremely thorough. One report described multiple health hazards encountered by 
the New Orleans Police Department (NOPD), documented symptoms, and identi- 
fied clear risk factors for a number of outcomes, including respiratory illness, 
post-traumatic stress disorder, and depression (NIOSH, 2006d). This report, pro- 
duced within seven months, includes 48 references and clearly identified findings. 
It is as a model for exploring emerging issues in disaster settings. At the time of 
this writing, the results from this report were in press in a pccr-revicwcd publica- 
tion and a follow-up field investigation was requested by the NOPD. These out- 
comes demonstrated both the quality of the new knowledge developed and the 
relevance of the findings to not only the appreeiafive requesting entity but also 
other public entities that may face similar disasters. 



RELEVANCE SCORE 

The program's strategic goals are appropriate, although some intermediate 
goats and performance measures could be made more ambitious if resources were 
available. The committee finds that the HHE Program responds well to HHE re- 
quests, and that HHEs play a key role in addressing widespread occupational 
health issues, such as musculoskeletal disorders, and emerging issues, such as 
lung disease from diaectyl-based flavorings. However, the committee believes the 
program could do a better job of eliciting a broader array of requests, especially 
from undcrscrved populations and those exposed to new or newly recognized 
hazards. Rapid changes in the economy and workforce have affected the nature of 
HHE requests. The HHE Program has generally met these challenges with a judi- 
cious use of its resources, while remaining true to its mission. The program could 
do even better by making greater use of available surveillance data and by sharp- 
ening its focus on emerging issues. 
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The HHE Program has established a triage process to determine how to best 
meet the needs of each requester within the limits of available resources. The tri- 
age process leads to improved efficiency, but it would benefit from increased 
transparency and more sophisticated site visit selection. Reports sent to the re- 
questers and other interested parfies and published on the NIOSH website are well 
written and relevant, reflecting a high level of expertise, and are supported by ap- 
propriate documentation. However, the quality and utility of some of the recom- 
mendations in the reports have not always met the same high standards. This 
could be corrected through a better quality assurance program. The timeliness of 
the reports has consistently been an issue in the past, but the HHE Program has 
shown some improvement in this area. 

The 9/11 attacks and Hurricanes Kalrina and Rita put needed emphasis on 
emergency response. The HHE Program has responded well to these emergencies 
and is uniquely qualified to develop better ways to protect the U.S. workforce 
during such disasters. NIOSH and the HHE Program are well prepared to deploy 
during emergencies. 

After consideration of the criteria provided in the Framework Document (see 
Box 1-2), the committee assigns the HHE Program a score of 4 for relevance. The 
committee was asked to evaluate whether program activities are in priority areas, 
and if the program is engaged in relevant transfer activities. Though tlie commit- 
tee makes multiple recommendations for improvement, the committee finds that 
program activities are in priority subject areas, and that the program is engaged in 
appropriate transfer activities. A lower score, according to the scoring criteria, 
would indicate either a focus on lesser priorities or a lack of information dissemi- 
nation. Had the committee not been required to give an integer score, they might 
have rated the program between a 4 and 5. 
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Impacts of the Health Hazard Evaluation Program: 
Reductions in Harm and Transfer of Knowledge 



DEFINITION OF IMPACT 

The committee is charged with evaluating the impact of the National Institute 
for Occupational Safety and Health (NIOSH) Health Hazard Evaluation (HHE) 
Program. Specifically, the committee is asked to determine the impact of the HHE 
Program on 

1. Reducing worker risk and preventing occupational illness in investigated 
workplaces; 

2. Transferring program-generated information to relevant employers and 
employees beyond the investigated workplaces; 

3. NIOSH research and policy development programs; and 

4. The activities of regulatory agencies, occupational safety and health pro- 
fessionals and organizations, state and local health agencies, and others in 
the occupational health community, as achieved by transferring program- 
generated hazard and prevention information. 

This chapter is organized into seven major sections. This section defines im- 
pact and the context by which the program is being evaluated. The next four sec- 
tions are analyses of each of the four types of impact described above, addressing 
both proximal and distai impacts of HHE Program activities, as well as program 
limitations. The committee then evaluates the impact of the HHE Program's 
emergency response activities. The final section of this chapter describes the 
committee score and rationale for program impact 

Ideally, a review of the HHE Program would yield quantitative evidence of 
the reduction of illness or death {earlier defined as "end outcomes") at specific 
investigated worksites or similar workplaces throughout the country. Such quanti- 
tative evidence of impact is generally unavailable. In the field of occupational 
health, however, there is solid evidence that exposure to certain chemical or 
physical agents causes illness, injury, or death. In general, a reduction in the level 
or frequency of exposures is expected to reduce the number of workers who de- 
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velop disease. Elimination of exposures can reduce the number of cases of disease 
in those already exposed, prevent disease in new hires, or both. Based on this 
cause-and-effect relationship, attempts are made to reduce disease risk by reduc- 
ing or eliminating exposures to various agents. Sometimes, based on reduced ex- 
posure, it is possible to make estimates of disease or deaths avoided. 

Reductions in exposure at one workplace as a result of HHE Program recom- 
mendations can lead NIOSH or other agencies to take action to reduce exposures 
in similar workplaces nationwide. Information regarding exposure reduction 
measures can be provided to employees at a site where an HHE was conducted, 
and then to occupational health professionals, public health practitioners, and state 
and federal regulators elsewhere through reports, hazard alerts, and other publica- 
tions. It is conceivable that NIOSH would have the data to estimate the number of 
lives saved and the reductions in diseases that can be direct or indirect results of 
HHEs conducted. For example, HHE recommendations describe the actions 
needed to reduce exposures of elevated chemical or physical hazards identified 
during an investigation. Follow-up with employers, and especially employees, 
could yield information about actions taken to reduce exposures. With sufficient 
evidence that harmful exposures were reduced or eliminated (for example, by 
substitution of one chemical or process for another), it may be possible to develop 
quantitative estimates of occupational illness or death avoided. 

Existing occupational health and safety data are insufficient to support robust 
analyses of impacts of the HHE Program. This is not to say that the HHE Program 
does not have impact, but that there are inadequacies in the reporting system. Be- 
cause occupafional disease and death statistics in particular arc limited in the U.S. 
health data systems, other evidence is examined by the committee to estimate 
HHE impacts. The committee began this process by looking at the HHE Pro- 
gram's strategic goals and determining the number of HHEs conducted relevant to 
each strategic goal and by hazard type (Table 4-1). Starting with a table provided 
by the HHE Program (NIOSH, 2007b:Table 3-1), the committee used the pro- 
gram's online HHE search engine to identify related reports.' This simple exer- 
cise yields somewhat different results with different search terms used for the 
same issues, but the exercise illustrates that the HHE Program likely has had im- 
pact in areas relevant to its strategic goals. The HUE Program provided the 
committee information on activities related to disease reduction, such as data sent 
to the Occupational Safety and Health Administration (OSHA) used for federal 
standard setting, which can be expected to result in lower exposures and, there- 
fore, expected reductions in disease. 



' See hitp://www.cdc,gov/niosh/hhe/ (accessed July 9, 2008) 

" The number of relevant IIIlEs idenlified by the search engine for a given search term varied over 

Ihe course of committee deliberations. 
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TABLE 4-1 Distribution of HHE Investigations by Strategic Goal and Hazard 



Strategic Goal 


Hazard type 


HHEs (numbered reports) 


I. Prevent occupational illnesses 


Biological 


Biosolids (1) 


through reduced exposure to 




Brucellosis (3) 


workplace hazards 




Latex allergy (7) 
Tuberculosis (46) 




Chemical 


Tertiary amines(3) 
Asphalt (37) 

Chlorinated compounds (28) 
Lead in construction (13) 




Physical 


Musculoskeletal hazards (77) 
Noise (244) 




Mixed 


Global health (0) 

Indoor environmental quality 

(200) 

Metalworking fluids (23) 


2. Promote occupational safely 




Flock, respiratory (7) 


and health research on emerging 




Flavorings (10) 


issues 




Severe Acute Respiratory Syn- 
drome (0) 
Silica in roofing (9) 
Carbon monoxide and house- 
boats (5) 

Surface wipe methods for 
chemical decontamination (1) 


3. Protect health and safety of 




Anthrax (2) 


workers during public health 




Irradiated mail (1) 


emergencies 




Natural disasters (0) 
Hurricane Katrina (3) 
World Trade Center (3) 



SOURCE: NIOSH (2007b; HHE 
2008]). 



1. HEALTH HAZARD EVALUATION PROGRAM IMPACT ON 

REDUCING WORKER RISK AND PREVENTING OCCUPATIONAL 

ILLNESS IN INVESTIGATED WORKSITES 

Finding 1.1: HHEs generally have a positive impact on improvement of 
occupational health at investigated worksites. 

There are many relevant examples of HHE reports that contain information 
useful to reasonably estimate reductions of health risk in the workplace. Numer- 
ous examples exist of workplace changes that have occurred as a result of HHEs. 
These include sotne fairly minor alterations in processes, or in the use of control 
technologies or housekeeping procedures (for example, hazard-specific clean-up 
procedures) that reduce exposures and possibly resulting in a rapid reduction in 
illnesses. Table 4-2 illustrates several examples of impact-producing HHEs con- 
ducted from 1978 through 2006. 
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TABLE 4-2 Examples of HHEs Resulting in Wide 


Impacts (1978-2006) 


Hazard 


HHE Report" 


Impact 


Di bromoc h loropropane 
(DBCP) 


- HHE-77-103-474 (Occidental 
Chemical Company, Lathrop) 

- HETA-78-004-1511 (Shell 
Chemical Company. Axis, 
Alabama) 

- HETA-8l-040-l3]5(Dole 
Pineapple Corporation. Lanai, 
Hawaii) 

- HETA-8l-l62-1935(Maui 
Land and Pineapple Inc., Ka- 
huti, Hawaii) 


- DBCP, once widely used as a nematocide and 
has been found to cause sterility among agri- 
cultural workers. 

- HHE Program data were used by OSHA to 
promulgate a Standard in 1979 (U.S. Depart- 
ment of Labor, 2008a) to require reduced ex- 
posures. 


Lead 


- 337 HHEs were conducted 
between 1978 and 1995 in a 
wide array of industries 


- HHE Program data were used by OSHA in 
promulgating lead standards (U.S. Department 
of Labor, 2008a, b). 

- For 25 years, HHEs provided information re- 
garding exposures and control measures used 
by evaluated industries, OSHA consultation, 
and enforcement activities. 


Silica/Roofmg 
Tiles/Construction 


- HETA-2003-0209-3015 (Di- 
versified Roofing Inc., Phoe- 
nix, Arizona) 

- HETA-2005-0032-2985 (Pe- 
tersen-Dean Roofing Sys- 
tems, Phoenix, Arizona) 

- HETA-2005-003l-3055(C& 
C Roofing, Phoenix, Arizona) 

- HETA-2005-0030-2968 
(Headlee Roofing, Mesa, 
Arizona) 


- NIOSII Publication 2006-110 (NIOSH, 
2006g), based on 4 HHEs, describes the haz- 
ard, silicosis, and how to protect workers. The 
Center for Construction Research and Training 
(CPWR)'' uses thi,s publication in a curriculum 
to train 20,000 roofers (P. Stafford, CPWR 
Executive Director, personal communication, 
April 2008). 


Diacetyl 


- See Box 3-2 for HUE Hislory 
1985-2006 


- Recognition of a pulmonary disease caused by 
diacetyl. 

- Publications in public health literature. 

- Data currently being used by CALOSHA to 
develop a standard to protect workers. 

- ConAgra, a major manufacturer, initiated pul- 
monary surveillance and industrial hygiene re- 
views to reduce exposure {Lockey, M.D,, J,E,, 
Professor, University of Cincinnati, written 
communication, February 18, 2008). 

- States alerted pulmonary physicians to diacetyl 
and bronchiolitis obliterans. 

- These HHEs caused NIOSH's pulmonary re- 
search unil to perform extensive research re- 
garding bronchiolitis obliterans and give this 
occupational hazard international attention. 

- Voluntary removal of diacetyl from manufac- 
turing processes in some facilities. 
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Silica Flour 



HHE-78-104-107 (Tamnisco 
Incorporated, Tamms, Illi- 
nois) 

HHE-79-103-108 {Illinois 
Mineral Company, Eico, Illi- 
nois) 



- Recommended exposure controls used in U.S. 
DOL Mine Safety and Health Regulation, 
1 980 {U,S, Department of Labor, 2008c). 

- Proceedings American Society for Testing and 
Materials, Spain, 1979. 

- N lOSI t Current Intelligence Publication 
(NIOSH, 1981) recommended to Indian Silica 
Flour Industries by C. Rice, Deputy Director, 
and S. Clark, Director, Education and Re- 
search Center, Department of Environmental 
Health. University of Cincinnati. 

- HHE site recommended to Dr. Amal El-Safty, 
Cairo University, Cairo, Egypt, 2008, by C. 
Rice. 



Synthetic Fibers/Flock 



See Box 3-3. 8 HHEs were 
conducted between 1972 and 
2004: 

HETA-96-0093-2685 
(Microfibres Inc., Pawlucket, 
Rhode Island) 
HETA-98-02 12-2788 
(Claremonl Flock Corpora- 
tion, Claremont, New Hamp- 
shire) 

HETA-98-0238-2789 {Spec- 
tro Coating Corporation, 
Leominster, Massachusetts) 
HETA'2004-00 1 3-2990 
(Hallmark Cards, Inc., Law- 
rence, Kansas) 
HETA-2004-0I86-3011 
(Claremont Flock Coipora- 
tion, Claiemont, New Hamp- 
shire) 

HHE-77-1 14-529 (The Stan- 
dard Products Company, 5 1 
Henry Clay Boulevard, Lex- 
ington, Kentucky, September, 
19781 

HHE-72-33-129 (Barker 
Greeting Card Company, 
Cincinnati, Ohio) 
HHE-80-214-799(MandB 
Metal Products, Inc.. Leeds, 
Alabama) 



High-hazard HIIEs and NIOSH follow-up re- 
search are only warnings. OSHA does not have a 
regulation. 
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Musculoskeletal Dis- 
orders 


Numerous HHEs from 1980- 
2007 (See Box 3-4) 


The HHE Program identified hazards causing 
disorders of muscles, tendons, and joints in di- 
verse industries with 50% in manufacturing. 
These HHEs led to reviews, practice documents, 
and have informed OSllA and other regulatory 
bodies in rulemaking (for example. U.S. Depart- 
ment of Labor, 2000). The HHE Program's body 
of work on this topic has stimulated major re- 
search activities within and outside of NIOSH on 
work-related musculoskeletal disorders. NIOSH 
has used these HHEs as bases of other investiga- 
tions among underreported populations (farm 
workers). 


Metal Working Fluids 
(MWFs) 


The HHE Program website indi- 
cated 19 HHEs were published 
between 1981 and 2006' 


An array of industries using MWFs, from aircraft 
and automobile plants to missile and hydraulic 
plants. NIOSH estimates 1.2 million American 
workers are exposed (NIOSH, 1998b). Exposures 
are associated with a range of illnesses, from 
dermatitis to asthma and other pulmonary effects. 
A publication co-authored by HHE staff de- 
scribed an investigation and associated hypersen- 
sitivity pneumonitis (HP), a rare but severe condi- 
tion, with MWF exposure (MMWR, 1996). An- 
other HHE identified a severe outbreak of HP at a 
small plant and was instrumental in abating expo- 
sures at the facility and quelling the outbreak 
(NIOSH, 2002d). Both investigations were pub- 
lished in a short time in Morbidity and Morality 
Weekly Report. HHEs have been important in 
defining the effects of MWFs and providing con- 
trol technologies in hazardous workplaces where 
no OSHA standards or guidelines have been es- 
tablished to control exposures. 


Latex Allergy 


Numerous HHEs from HHE 
Website 


NIOSH identified powdered latex gloves as the 
risk factor for latex allergy. Massive adoption of 
powder-free latex gloves followed. 



"Specific HHE Program reports mentioned in this table can be accessed at the HUE Program website 

(http://www.cdc.gov/niosh/hhe/, accessed July 25, 2008). 

''Formerly known as The Center to Protect Workers' Rights. 

''See http;//www.cdc.gov/niosh/hhe/HHEprogram,html (accessed .luly 25, 2008) 



The HHE Program tracks data about HHE requests through a management sys- 
tem, including the approximate number of people exposed to hazards (NIOSH, 
2007g). Although this system has only been in plaee since 2005, HHE Program 
staff estimate that nearly 400,000 employees have been at risk at sites where 
HHEs have been requested between fiscal years 2005 and 2008. This is based on 
information received from 87 percent of all requestors (information is missing for 
the remainder of the requests) (NIOSH, 2007h). The management tracking system 
eould be improved with greater follow-up to obtain initially unreported data, and 
with better estimates of the exposed worker population in field-investigated sites. 
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A previous program evaluation by the Research Triangle Institute (RTl, 1997) 
recommended the HHE Program systematically survey employers and employees 
in workplaces where HHEs, technical assistance, or consultation were provided to 
learn whether HHEs satisfied customer needs, resulted in improved workplace 
health and safety, and identified emerging problems. As described earlier in this 
report, the HHE Program may conduct a followback survey at the completion of 
an HHE. One program staff person is responsible for followback survey activities. 
The survey response rate is about 60 percent (NIOSH, 2007b). 

Among followback survey rcsponders (289) who indicated they knew about 
the actions resulting from an HHE, 62 percent reported actions to implement HHE 
recommendations, and another 12 percent reported that actions were planned. 
Corrective actions taken were primarily housekeeping (85 percent), whereas per- 
sonal protective equipment, engineering or administrative controls, and exposure 
monitoring varied from 50 to 75 percent of the remainder. In a survey of 68 re- 
spondents, 62 percent reported employee health had improved (NIOSH, 2007b). 
This information, while not rigorously confirmed as representative, is important 
evidence in evaluating whether HHEs reduce risks in the workplace. 

Recommendation: The foUowback surveys should be modified for use in as- 
sessing the relevance, feasibility, and impact of recommendations made by 
the HHE Program. 

Finding 1.2: The number of HHE investigations completed in the last 
decade has declined dramatically. Therefore, fewer sites receive the posi- 
tive benefits that accrue as a result of completed HHE investigations. 

The HHE Program received an average of 372 requests for investigations per 
year between 1997 and 2006 compared with a historic average of 498 per year 
prior to 1997. The number of field investigations declined during the past decade 
from 126 in 1997 to 58 in 2006 (NIOSH, 2007b). One reason for the decrease in 
field investigations was an increase in the proportion of requests concerning in- 
door environmental quality (lEQ) issues which could be addressed by telephone 
calls and letters. In the same period, invalid requests increased from 15 to 46 per- 
cent. 

One can speculate that reductions over the past 10 years in both total and valid 
HHE requests have resulted from the loss of jobs in the industrial sector, fewer 
health and safety personnel among union and non-union industries, or fewer haz- 
ards in workplaces. The number of full-time equivalents (FTEs) on staff has 
dropped from 63 in 1998 to 52 in 2006 (NIOSH, 2007b). The budget for travel to 
perform HHEs has remained essentially the same over the period from 2000 to 
2007 at the NIOSH Hazard Evaluations and Technical Assistance Branch 
(HETAB) (NIOSH, 2007b), indicating less money is available for field investiga- 
tions given increases in travel costs. Not only docs the budget create problems for 
hiring adequate numbers of personnel, but the infrastructure (such as equipment 
and physical infrastructure) needed for HHE field investigations is eapilal inten- 
sive, potentially limiting the number of field investigations. 
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The reasons for the drop in field investigations are difficuU for the eommittee 
to assess with confidence. However, the number of completed HHEs is reduced. 
As a result, the potential impact of HHEs could also be reduced. The committee 
concludes there likely are budgetary constraints in terms of personnel and equip- 
ment necessary to carry out HHEs given the decrease in FTEs and relatively fiat 
budget between 2000 and 2007. 



Finding 1.3: Large portions of the labor force — particularly those from 
traditionally underserved populations — are unaware of HHEs. Hence, 
they do not request the service, and are unaware that a substantial body 
of HHE work exists to assist them. 

One factor that may contribute to the decrease in HHE requests is that rela- 
tively fewer workers may be knowledgeable about the role HHEs can have in 
making their jobs safer. The structural shifts in the labor force away from union- 
ized settings and toward the service sector, as well as the increase in contingent 
workers have not been addressed by the HHE Program with additional outreach 
resources. The HHE Program lacks personnel to provide outreach to these work- 
ers who may benefit from HHEs. As an agency, NIOSH is strong in its abilities to 
conduct outreach when funds are available (for example, successful outreach to 
former nuclear workers funded by the U.S. Department of Energy through 
NIOSH) (NIOSH and OERP, 2001). In response to the committee's request for 
stakeholder input (see Chapter 1 and Appendix D), many respondents indicated 
they did not know about the HHE Program. Others indicated they knew about the 
program and suggested outreach methods. At a December 2007 Worksafe Con- 
ference in California, several community/worker groups indicated no knowledge 
of what an HHE was or how to request one (Worksafe, 2007).^ Recommendations 
from stakeholders about how to better communicate with these populations arc 
included in the closing section of this chapter and Appendix D. 

A recent journal article by NIOSH investigators Cummings and Kreiss (2008) 
described the potential health risks of contingent workers, noting there may be a 
lack of health and safety training and limited personal prolcctivc equipment for 
them. Contingent workers have nontraditional employment relationships; for ex- 
ample, they may be temporary or part-time workers, or independent contract 
workers employed in refineries, agriculture, mining, or construction. The authors 
note these workers are frequently young, female, African-American, or Hispanic, 
and have lower incomes and fewer benefits. This assessment was also made dur- 
ing a stakeholder presentation at the committee's second meeting (Gittleman, 
2007). An exception to this characterization is construction workers who are 
members of the national building trades unions. These workers typically receive 
health and safety training and understand their rights. 



^ Worksafe is a California-based nonprotit organization established to promote occupational safety 
and health through education, training, technical and legal assistance, and advocacy. Members 
include labor and community groups, workers, occupational safety and health and other profes- 
sionals, environmenlali.sts. and others (htlp;//w ww.worksafe.org/, accessed March 30, 2008). 
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During presentations to the committee, HHE Program staff acknowledged the 
need to give attention to shifts in demographics and composition of the labor 
force. There may be sectors and employment groups not being served by HHEs 
that should be, and the committee suggests that day laborers, immigrant groups, 
some service industry workers, and low-wage workers may be examples. The 
HHE Program has had some success in reaching such underservcd populations 
(see Box 1-1 for definition of underserved populations). An HHE conducted for 
the Alameda County, California, Public Authority for In-Home Support Services, 
for example, led to successful and important HHE impact on a new kind of em- 
ployment group — the home healthcare workers the county office is serving 
(NIOSH, 2004a). The state contracts with the agency to use local independent 
contractors for home healthcare. Worksites arc as varied as the clients needing 
home healthcare, and the workers are not in contact with one another. This par- 
ticular HHE is also an excellent example of leveraging resources with state and 
local partners: a successful partnership among the federal, state, and local em- 
ployment partners was necessary to conduct the HHEs, and innovative ways of 
communicating the results to multicultural workers working in multiple sites was 
required. 



2. TRANSFER OF PROGRAM-GENERATED INFORMATION TO 
RELEVANT EMPLOYERS AND EMPLOYEES BEYOND 
INVESTIGATED WORKPLACES 

Finding 2,1: The HHE Program disseminates its findings, sometimes widely. 

Based on discussion with the HHE Program staff, program information is dis- 
seminated to employers and employees beyond the investigaled workplace 
through 

• HHE reports on the Internet accessed through the NIOSH website; 

• CD-ROMs of HHEs available free to industries, workers, professors, and 
students; 

• notification of subscribers by e-mail about various types of free docu- 
ments; 

• appropriate OSHA regional offices; 

• scientific articles in tcclinical journals; 

• NIOSH documents such as Health Hazard Alerts released when a new 
body of information requires a special report to bring to relevant commu- 
nities (approximate 1 to 5 times per year); 

• articles in trade publications; 

• presentations at conferences and workshops; and 

• alerts to other government agencies, including OSHA, at federal and state 
levels that may be in contact with employers and employees at similar 
workplaces. 
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Listed above are examples of mechanisms to transfer program-generated in- 
formation to relevant employers and employees. Figure 4-1 is a flow diagram 
showing the potential transfer paths of HHE Program outputs to stakeholders and 
potential impacts they may have in general terms. HHE reports themselves, 
searchable on the NIOSH website, are important vehicles for affecting workplaces 
other than those investigated, given appropriate transfer mechanisms. CDs of 
HHE reports have been prepared, when funds were available, and provided free of 
charge to students in training in industrial hygiene, in occupational medicine and 
nursing, and as industrial safety specialists. Professionals in the field (including 
members of this evaluation committee) have also received such CDs. Members of 
the NIOSH Education and Research Centers (ERCs) training programs in masters 
and doctoral programs regularly use information collected in the conduct of HHEs 
(Carol Rice, Deputy Director, Education and Research Center, personal commu- 
nication, April 2008). NIOSH has relationships with the International Labor Or- 
ganization and the World Health Organization, and shares hard copy and elec- 
tronic documents for distribution to workplaces internationally (Paul Schulte, Di- 
rector, Education and Information Division, NIOSH, personal communication, 
April 2008). The committee encourages relationships with these groups and with 
international NIOSH counterparts. 

Because there are no U.S. Department of Labor regulations in place covering 
the use of many of the hazards identified in HHEs, these documents are valuable 
resources for employers, employees, consultants, and educators seeking to reduce 
illness and death in the workplaces (see Tabic 4-2 for examples such as those re- 
lated to MWFs and musculoskeletal disorders). Annual reports, which might in- 
clude information on the numbers and categories of HHE requests, industries and 
hazards involved, types of responses, resources required for responses, summary 
of key findings fiom selected HHE reports, data on fimeliness goals, and out- 
comes and impacts, might also prove useful tools for disseminating information 
about the program and its activities. 



Finding 2.2: There is variable penetration of information into some 
communities. 

Table 4-2 illustrates industries in which there has been widespread dissemina- 
tion and transfer of informafion to relevant employers and employees. There is 
little to suggest, however, that HHE-gencratcd information is received by em- 
ployers and employees in small workplaces or by members of underserved popu- 
lations. For this reason, the committee necessarily relied largely on anecdotal in- 
formation. Below, the committee cites three examples of incomplete dissemina- 
tion of HHE Program information. 
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Example 1: Trade Organizations 

The committee heard from the vice president of manufacturing of a small flavor and 
fragrance manufacturing company in California (Speakman, 2008). An HHE was con- 
ducted at his facility to determine the cause and prevention of pulmonary disease among 
workers (NIOSH, 2007J). The company representative had not heard of the flavoring- 
related illness prior to the investigation at his worksite. The committee noted a relevant 
trade association was already aware of the hazard at the time the HHE was conducted. 
The representative infonned the committee that he was not a member of the particular 
trade association from which he may have gotten relevant information, but. because of 
cost, was a member of a different trade association. Discussion led the committee to con- 
clude that reliance on trade associations as a means for the HHE Program to reach small 
business may not be sufficient. 

After review of the list of HHE Program presentations made at trade conferences pro- 
vided to the committee (NIOSH, 2007b:Appendix 2.14), the committee agreed that ap- 
pearances in such venues are important and useful. However, the committee also received 
verbal and written stakeholder input that the HHE Program may need to increase its range 
of trade- and business-related venues. This is discussed in greater detail in Section 4 of 
this chapter. 



Example 2: Small Residential Construction Companies 

A representative of the Center for Construction Research and Training (CPWR)^ 
noted that HHE Program success stories include HHEs that have resulted in better control 
technologies to reduce exposures to, for example, lead and silica (Gittleman, 2007). 
These HHEs were conducted primarily in union settings. Because of the mobile and con- 
tingent nature of the construction industry, however, many workers may still be exposed 
to risks that arc well understood. The CPWR representative described a need to target 
high-risk residential construction in small companies that do not have the resources to 
investigate and solve their work-related safety and health problems. 



Example 3: The Immigrant Labor Workforce 

A stakeholder spoke to the committee on behalf of the Interfaith Worker Justice Na- 
tional Workers' Centers Network" and noted a general lack of knowledge and under- 
standing among the immigrant population it represents of what the HHE Program docs, 
how it differs from the Department of Homeland Security's Immigration and Customs 
Enforcement (ICE) office, and how the HHE Program can benefit workers (Oliva, 2007). 



'' The California Department of Public Health requested the ItllE at the facility in 2006 with the company's 
cooperation, when two cases of pulmonary disease were identified at the facility. The HHE resulted in two 
additional cases of the illness being identified (NIOSH, 2007k), 
^ Foniierly known as the Center to Protect Worker Rights, 

" The Interfaith Worker Justice is a network of people of faith established to educate, organize, and mobi- 
lize the religious community to act to protect the rights of, and improve working conditions and benefits, 
for workers, especially low-wage workers {http://iwj.org/, accessed June 1 , 2008). 
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The inability of workers' centers to file HHE requests on behalf of workers, as well as the 
fear of government and negative experiences indicate the need for special outreach to 
worker centers, community-based and faith-based institutions, and day labor organiza- 
tions. The need for materials to be translated and to transfer information in culturally ef- 
fective ways was also stressed by this individual. 

In general, the committee finds evidence that HHE findings are disseminated widely 
to occupational health professionals, industries represented at technical meetings, certain 
trade associations, and a portion of the general public health community and worker 
training programs. The HHE Program stated in a written response to questions from the 
committee that it does not receive many requests from the agriculture and construction 
sectors. This was attributed to the temporary nature of the work, the mobility of the labor 
force, and the immigrant status of large parts of this labor force (NIOSH, 2007h:question 
#22). HHE Program staff members also indicated their awareness of the need to extend 
outreach, especially among the contingent workforce. The HHE Program has no formal 
mechanism to evaluate the effectiveness of knowledge transfer to employers and employ- 
ees in facilities that have not been investigated. 

Boxes 3-2 and 3-3 describe the occupational hazards of diacetyl and flock, respec- 
tively, and the HHE Program's essential role in determining the relationship between 
worker exposure to these substances and serious respiratory diseases. However, the pub- 
lic and policy responses to these two substances have been completely different. There 
has been widespread media and regulatory attention focused on diacetyl, while flock has 
continued in production with relatively little attention. In some respects, the attention to 
diacetyl may be due to potential exposure among the general public as well as to workers. 
The HHEs related to diacetyl hazards were an important part of a chain of events that will 
likely result in rulemaking and has already led to limited voluntary substitution with other 
flavoring agents. In contrast, the HHEs related to flock exposure generated recommenda- 
tions by NIOSH, but no attention at OSHA. The extent to which the flock industry fol- 
lows the NIOSH guidcHncs is not known to the committee. It may be instructive to the 
HHE Program to assess the differences between the public responses to diacetyl and 
flock, and to use these findings to improve information dissemination for similar issues in 
the future. 

Recommendation: Develop a proactive, comprehensive information-transfer strat- 
egy for HHE Program outputs with better approaches to reaching wider audiences, 
including traditionally underserved populations. The HHE Program could 

a. Use innovative techniques to reach small businesses and underserved popula- 
tions, creating a broad array of mechanisms for communicating with diverse 
constituencies and attending to issues of literacy, language, and national- 
origin barriers. 

b. Develop distribution mechanisms that are not Internet-dependent to com- 
plement Internet distributions. 

c. Disseminate HHE results more broadly to groups likely to be affected, in- 
cluding distribution of HHE reports in the geographical regions where inves- 
tigations are conducted. 
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d. Increase efforts to consolidate findings of multiple HHEs for specific hazards 
(for example, the compendia compiled for lead and other topics). 

e. Develop improved methods of outreach to stakeholders so that workers and 
workplaces affected by new and emerging occupational health problems will 
be alerted quickly. 

f. Supplement program outreach efforts by using community and small-business 
groups to translate HH£ results and findings for their constituencies. 

g. Leverage existing NIOSH, Centers for Disease Control and Prevention 
(CDC), and Department of Health and Human Services (HHS) resources to 
enhance technology transfer. 

h. Evaluate the effectiveness of information transfer programs, including 
knowledge transfer to employers and employees not investigated. 



3. IMPACT OF THE HEALTH HAZARD EVALUATION PROGRAM ON 
NIOSH RESEARCH AND POLICY DEVELOPMENT PROGRAMS 

The HHE Program is a relatively small unit within NIOSH, which itself is a relatively 
small unit within CDC, which is a relatively small unit of the HHS. A program of such 
small size and such a diverse mission needs to use the resources of its parent agencies to 
fulfill some of its functions. Furthermore, because the HHE Program itself does not have 
regulatory authority, its ability to have larger policy impacts is a function of its integra- 
tion into the policy communication and decision-making structures of the agencies au- 
thorized to engage in policy development. In brief, the abihty of the HHE Program to 
have impacts on other related research portfolios and policy development must in part 
flow through higher-level policy chains to have an impact on worker safety at the macro 
level (for example, beyond that of the individual HHE site level). This section of the re- 
port is divided into findings related to policy impact and findings related to impact on 
research programs. 



Policy-related impact 

Finding 3.1: The HHE Program does not have the authority to promulgate regu- 
lation but does inform the setting of regulations. 

The HHE Program has had substantial impact on policy and the development of regu- 
lation to protect workers in several high-profile areas. Furthermore, in several high- 
profile areas, the HHE Program contributed knowledge that led to changes beyond what 
was required by law or regulation on the part of industry. These changes have led to posi- 
tive impacts on worker health and safety. 

Policy is a multifaceted concept that is particularly complicated in the domain of pub- 
lic health. Federal policy making and implementation are ultimately the responsibility of 
elected or appointed officials in the executive, legislative, and judicial branches. The 
HHE Program is part of HHS, a large and diverse executive cabinet agency, which is su- 
perordinatc to CDC, which is supcrordinate to NIOSH, which in turn is superordinate to 
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the HHE Program. As with all cabinet agencies, the policy agenda is ultimately set by the 
President of the United States. External political and organizational factors govern the 
types of policy impact that the HHE Program can have and are the dominant factors lim- 
iting the program's ability to have policy impact. 

The HHE Program is quite small in both budgetary authority and personnel. Further- 
more, its legislative mandate has been interpreted — through regulation — as very narrow 
with respect to the definition of what constitutes a valid request. Chapter 2 describes the 
regulatory interpretation. In this chapter, the committee interprets how the HHE Program 
leverages the resources of other occupational heahh actors to perform its mission and ex- 
tend the impact of its activities and outputs. 

The committee found substantial evidence that the HHE Program is positioned to 
provide high-quality technical advice — when requested — with some policy implications 
at the state level, and for other parts of the federal government. For example, the HHE 
Program is represented on the National Occupational Research Agenda (NORA) sector 
councils and other federal occupational health committees with other agencies, such as 
OSHA, the U.S. Environmental Protection Agency (EPA), and the U.S. Navy {NIOSH, 
2007b). In one case, the HHE Program teamed up with the Federal Highway Administra- 
tion to evaluate asphalt exposure during paving (NIOSH, 2002a), and with the Transpor- 
tation Security Administration to assess radiation exposure by baggage screeners 
(NIOSH, 2006h). In general, it would appear that most of such efforts are targeted at the 
federal level, but it is beyond the scope of the committee's charge to evaluate the impact 
of these mechanisms outside the HHE Program. Discussions with HHE Program staff 
during committee meetings suggest that staff hold a narrow view of the HHE Program 
role in policy development, either within or outside of NIOSH. The impression is one of 
purposefully keeping a low profile. 

The committee has already noted the substantial evidence that supports the HHE Pro- 
gram's ccntrality in policy-relevant emerging hazard areas such as emergency response, 
and with hazards associated with diacctyl, MWFs, and latex gloves. During the course of 
this evaluation, the HHE Program became actively involved in study of the inflammatory 
neuropathy among swine slaughterhouse workers in Minnesota (MMWR, 2008). The 
committee considers this an example of how the HHE Program is positioned to respond 
quickly to emerging health hazards; at the beginning of this committee's deliberations, 
the issue was not of particular concern, but toward the end, the problem was nationally 
prominent. The findings emerging from HHE involvement arc being used to develop 
safeguards in such facilities, and may extend to other kinds of production involving high- 
pressure treatment of brain material. The committee concludes that the HHE Program has 
a very good reputation (among those who know about it) and that it is readily called upon 
in the event of new, emerging, and poorly understood problems. 

At the time this report was being prepared, there was substantial evidence that HHE 
Program activity on the topic of diacctyl (a butter flavoring) has led to significant legisla- 
tive efforts geared toward eliminating diaeetyl from production processes. For example, 
California Assemblywoman Sally Lieber (D) introduced a bill to ban diacctyl use by 
2009 (Cahfomia Assembly, 2007).^ In September 2007, the U.S. House of Representa- 



^ This bill was approved by the California Assembly, but was put into an inactive file of the California Sen- 
ate in February 2008. As of June 4, 2008, no new action had been taken on this bill (California Assembly, 
2008). 
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tives passed a bill to require OSHA to set a standard limiting diacetyl exposure (U.S. 
Congress, House of Representatives, House Education and Labor, 2007); as of June 2008, 
the bill was on the Senate Legislative Calendar. It is still not clear how the HHE Program 
participates in developing specific policy and research recommendations. For example, 
the committee was not able to determine exactly how the HHE Program originally be- 
came involved in the diacetyl issue. Diacetyl is used to flavor popcorn and a variety of 
other food products (see Box 3-2); however, it remains unclear how the HHE Program 
will expand knowledge about the hazards in the absence of HHE requests (for example, 
NIOSH, 2007J) or the very-high-profile cases related to popcorn manufacturing. This 
case perhaps highlights the difficulty the HHE Program has in proactively influencing 
policy development because it must be asked to conduct investigations or be called on by 
policy makers for advice. The program's enabling regulations limit its ability to be proac- 
tive in participating in the regulatory process. 

In an early case of high policy impact, in 1977. an HHE conducted in collaboration 
with the University of California developed evidence that dibromochloropropane (DBCP) 
caused sterility among workers at facility in California (NIOSH, 1977). Several months 
later, OSHA used these program- generated data and experimental carcinogenesis data to 
regulate this chemical. This reduced the risk of sterility in people exposed to DBCP. 
Since DBCP is a component of pesticides, the impacts are far reaching on a number of 
different industries and worker groups. These efforts also contributed to the placement of 
this chemical on the EPA pesticide registry. 

The body of MWF HHE reports had a significant input to the NIOSH Criteria for a 
Recommended Standard: Occupational Exposure to Metalworking Fluids (NIOSH, 
1998b), which recommended the first authoritative exposure limit for MWFs. The find- 
ings led an MWF Standards Advisory Committee to recommend an OSHA standard be 
set based on the NIOSH recommended exposure limit (REL), medical surveillance, and 
other provisions (Sheehan, 1999). OSHA declined to move forward with a standard, 
which is an external factor beyond the control of the HHE Program and NIOSH, limiting 
the impact of these findings. Nevertheless, the NIOSH REL, which is based in substantial 
part on these HHEs, has become the benchmark for equipment design and maintenance in 
significant industrial facilities. Committee members knowledgeable on this topic reported 
that new equipment manufactured in this industry comply with the NIOSH REL, and that 
substantial progress has been made retrofitting older equipment to achieve this exposure 
limit. 

The HHE Program played an important role in efforts to develop regulation related to 
indoor air quality. Ultimately, this policy process did not progress to an established regu- 
lation, but the ccntrality of HHE Program work in the deliberative process is indicative of 
its excellent reputation when higher levels of policy making arc introduced. 

An OSHA employee who spoke with the committee suggested that prioritizing rec- 
ommended controls in HHE reports would help OSHA to be more specific in standard 
setting (Kent, 2007). Overall, she highly commended the HHE Program, indicating that 
report findings are crucial for the regulatory process, but regrettably indicated that the 
HHE Program was not used to its fullest within OSHA or in industrial hygiene training 
communities. Ways of improving knowledge about the program are addressed in subse- 
quent sections of this chapter, and in Chapter 3. 
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The penetration of powder-free gloves into the healthcare market and practice is a di- 
rect result of HHEs conducted in healthcare delivery settings. Prior to HHE involvement, 
there was widespread use in the healthcare industry of powdered latex gloves, which 
caused a latex-related allergic reaction among some users. Several HHEs were con- 
ducted; as a body, these HHEs associated latex rubber protein with adverse allergic health 
reactions. The HHE investigators were among those who created the NIOSH Alert: Pre- 
venting Allergic Reactions to Natural Rubber Latex in the Workplace (NIOSH, 1997c). 
These were distributed very broadly within the healthcare-delivery sector. Healthcare 
professional organizations cited the alert in making recommendations to provide alterna- 
tives to powdered latex gloves (e.g., Nagel, 1997 [American Chemical Society]; New 
York State Department of Health, 1998; NIOSH, 2008c). 

Recommendation: Develop more extensive formal linkages and mechanisms with 
other parts of NIOSH, CDC, and HHS to enhance the capacity for involvement in 
policy relevant impacts through 

a. Promotion and increase in direct communication, especially with OSHA and 
state occupational safety and health agencies. 

b. Alerts to NIOSH and CDC about HHEs that are relevant to policy-making 
outside the CDC system. 

c. Continued regular use of the NORA sector councils and the NIOSH Board of 
Scientific Counselors to disseminate information about the HHE Program. 

d. Pursuit of a change in the HHE Program's legislative and regulatory author- 
ity to improve the capacity to identify hazards in need of HHEs, improve the 
ability to gain entrance to facilities when requested by treating physicians or 
community representatives, and address exposures other than chemical 
agents. 



Finding 3.2: The HHE Program provides data and personnel to support NIOSH- 
recommended guidelines and NIOSH policy development activities. 

The HHE Program communicates occasionally with other policy-related organiza- 
tions, including those higher in the NIOSH hierarchy, to develop and forward policy rec- 
ommendations. This activity may be limited by the organizational design of the agency 
(matrix management), the HHE Program's narrow interpretation of policy-relevant prob- 
lems, and political concerns. The program has made contributions to NIOSH policy de- 
velopment through interactions with the NIOSH Authoritative Recommendations (AR) 
Program. According to written response to questions from the committee, the HHE Pro- 
gram has provided staff to support AR activities, including the preparation of written and 
oral testimony in support of proposed OSHA standards, and has provided data to support 
NIOSH policy development (NIOSH, 2008f). The HHE Program provided the committee 
information on 15 different areas in which the HHF Program has made or is making con- 
tributions toward NIOSH standards or policy developments (see Table 4-3). 

The participation of the HHE Program in AR Program activities is evidence of how 
NIOSH fulfills its policy functions using HHE Program research. E.vamples of NIOSH 
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policy functions include providing testimony to Congress, participating in OSHA regula- 
tory processes, publishing guidance documents, and developing public statements for dis- 
semination on the Internet. The HHE Program provides valuable support to these policy 
activities, which are outside the HHE Program regulatory mandate, but within that of 
NIOSH. 



TABLE 4-3 HHE Program Interaction with the NIOSH Authoritative Recommendations 
Program 



Tuberculosis 


HHE investigators and data supported NIOSH testimony related to proposed 
OSHA rule on occupational tuberculosis 


Ergonomics 


• HHE investigators and data supported NIOSH testimony related to proposed 
OSHA rule on ergonomics (www.cdc.gov/niosh/docs/2001-108/pdfs/200l- 
108.pdf) 

• HHE investigators and data supported NIOSH guidance document 
(http://www.cdc.gov/niosh/97-l I7pd.html) 

• HHE investigators and data supported NIOSH science review document 
(http://www.cdc.gov/niosh/docs/97-l4l/) 


Biosolids 


HHE investigators and data supported NIOSH guidance document 
(hitp://www.cdc.gov/niosh/docs/2002-149/pdfs/2002-149.pdf) 


Latex 


HHE investigators and data supported NIOSH guidance document 
(http://www.cdc.gov/niosh/latexalt.hlml) 


Indoor environ- 
mental quality 


HHE investigators, drawing on their HHE field experience, helped develop 
NIOSH testimony to the U.S. Department of Labor on indoor air quality 
(Rosenstock, 1 996) 


Metalworking flu- 
ids 


HHE investigators and data supported NIOSH policy document 
(http://www.cdc, gov/niosh/98- I02.html) 


Take-home lead 


HHE investigators and data supported NIOSH policy document 

(http://www.cdc.gov/niosh/contamin.html) 


Histoplasmosis 


HHE investigators and data supported NIOSH guidance document 

(http://www.cdc.gov/niosh/docs/2005-109/) 


Hexavalent chro- 
mium 


HHE investigators and data supported NIOSH policy statement: NIOSH 
Comments on the OSHA Request for Information on Occupational Exposure to 
Hexavalent Chromium 
(http://www.cdc. gov/niosh/topics/hexchrom/pdfs/Cr(Vl) NIOSH OSHA.pdf) 


Respirator decision 
logic 


HHE data supported NIOSH policy document 
(http://www.cdc,gov/niosh/docs/2005-100/appendix.htnil) 


Hearing loss 


HUE investigators supported NIOSH guidance document 
(http://www.cdc.gov/niosh/docs/96-l lO/pdfs/96-l lO.pdf) 


Body art 


HHE data supported NIOSH guidance 
(http://www.cdc.gov/niosh/topics/bbp/bodyari/) 


Workers with dis- 
abilities 


HHE investigators and HHE data helped develop NIOSH website content 
(http://www, cdc.gov/niosh/topics/wdd/default. html) 


Emergency re- 
sponse 


HUE investigators, drawing on their HHE field experience, helped develop 

NIOSH website content (http:/ywww. cdc.gov/niosh/topics/emres/) 


Skin notation 


In progress 



SOLfRCE: NIOSH (20081). 
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Finding 3.3: Relationships with certain state and local health departments ap- 
pear to be strong. 

The committee heard from stakeholders of important relationships between the HHE 
Program and certain local and state health departments, but the committee also learned of 
some state and local governments with which the HHE Program does not sufficiently 
communicate. The HHE Program has participated in NlOSH-sponsored surveillance 
meetings to reach state surveillance programs. It has made presentations to state-based 
organizations (for example, the Council of State and Territorial Epidemiologists), and has 
made contact with specific states. It has used the CDC Epidemic Intelligence Service 
(EIS) program to reach some state health departments, and has trained state-based EIS 
officers at the Morgantown facility. It would not be expected that the HHE Program 
would work closely with the NIOSH FataHty Assessment and Control Evaluation 
(FACE) program because of FACE Program focus on acute traumatic fatalities as op- 
posed to occupational illness. 

Despite strong outreach efforts made in a general way and very good relationships 
developed in some specific cases (for example, with the California OSHA), outreach to 
state and local health departments is incomplete. Communicating with state health de- 
partments may not translate to communication with local health departments. Responses 
to the committee's request for stakeholder input indicate similar concerns about outreach 
to local health departments (sec Appendix D for a summary of responses). 



Research-Related Impact 

In this section, the committee focuses on two aspects of research impact: those related 
to the development of laboratory or field-related experimental research programs at 
NIOSH and elsewhere, and those related to the impact on training the occupational health 
labor force. Section 4 of this chapter examines scientific publication impact specifically. 
Overall, for a program that is characterized as a service program, the committee found the 
HHE Program to be quite actively engaged in the applied scientific enterprise. 

Finding 3.4: The HHE Program strongly influences research programs and the 
scientific body of knowledge. 

The HHE Program has both indirect and direct influence on the research directions of 
NIOSH programs. For example, the various NORA committees develop research agendas 
for NIOSH programs they advise. According to a NORA sector committee member, 
some of the agendas arc influenced by the work of the HHE Program (Barbara 
Silvcrstein, Research Director, Safety and Health Assessment and Research for Preven- 
tion, Washington Stale Department of Labor and Industries, personal communication, 
July 9, 2008). More directly, much of the research within, for example, the NIOSH Res- 
piratory Diseases Research Program (RDRP), is based on unanticipated workplace haz- 
ards identified through the HUE Program. The HHE Program has stimulated research in 
areas such as the respiratory problems caused by flavorings, flock, waterproofing spray, 
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and vaporized viruses. The output of this research is evidenced by the extensive publica- 
tion list described in Section 4 of this chapter. 

Aside from impacts already highlighted in this report, the HHE Program, since the 
1990s, has also contributed to other research areas, including the engineering control of 
noise exposure in indoor firing ranges; the etiologic relationships and health effect 
mechanisms associated with various musculoskeletal disorders and with male reproduc- 
tive function associated with bicycle seats; and exposure monitoring techniques related to 
environmental tobacco smoke, nanotechnology, anthrax, and abrasive blasting (NIOSH, 
2008f). Though it was beyond the committee's charge to review other NIOSH programs 
influenced by the HHE Program, the committee assumes, based on information received 
from the program, that it has influenced research within the NIOSH Cancer, Reproduc- 
tive and Cardiovascular Diseases Program, the Engineering Controls Program, the Expo- 
sure Assessment Program, the Hearing Loss Prevention Program, the Musculoskeletal 
Disorders Program, the Nanotechnology Program, and the Personal Protective Technol- 
ogy Programs, among others. The committee was not able to determine the level of influ- 
ence based on the information presented. It is evident that HHE Program input has led to 
major contributions of new information in the literature and increased the scientific 
knowledge base about workplace hazards. 

One of the strengths of the HHE Program is its flexibility and responsiveness to a va- 
riety of occupational health problems. It maintains this strength with a very small budget 
and limited personnel, which requires the effective use of scientific resources located 
elsewhere. For example, NIOSH has an interagency agreement with the HHS National 
Toxicology Program (NTP) to support research on complex industrial exposures (Bucher, 
2008). NTP evaluates chemical agents of public heath concern for chronic toxicity, in- 
eluding carcinogenicity and reproductive toxicity, using modem methods in toxicology 
and molecular biology (NTP, 2008). Chemicals to be tested may be nominated by anyone 
with a concern. In 1998, the HHE Program conducted two HHEs related to bromopro- 
pane, a group of chemicals for which there are no OSHA standards, at the request of the 
North Carolina Department of Occupational Safety and Health (NIOSH, 2002b, 2003d).^ 
Comprehensive study of these chemicals would have required vast resources, and NIOSH 
subsequently nominated bromopropane in two chemical forms for evaluation by NTP. 
NTP evaluated bromopropane based on documented evidence of worker exposures (cit- 
ing HHEs in its final reports) and the published evidence of reproductive and develop- 
mental toxicity, and defined the erifical needs for data to improve assessment of toxicity 
to humans (NTP, 2002a, b).^ HHE Program investigations led to two other published 
studies on bromopropane, one assessing the relationship between DNA damage and bro- 
mopropane exposure, and the other to better characterize the bromopropane exposure 
hazard and to evaluate the utility of a biomarker for assessing exposure (NIOSH, 2007b). 
NTP sometimes calls on NIOSH for its experfise to help evaluate the urgency of testing 
chemicals nominated by others for review. For example, the HHE Program may conduct 



** Bromopropane is used in spray adhesives, as a precision cleaner and degreaser, and in the synthesis of 
pharmaceuticals, insecticides, quaternary ammonium compounds, and flavors and fragrances, and as a sol- 
vent for fals, waxes, or resins {NTP, 2002a, b). 

'' The NTP concluded that sufficient evidence existed to characterize 2-bromopropane as a reproductive 
hazard for hnmans {NTP, 2002b). 1 -Bromopropane was determined to be a reproductive hazard for labora- 
tory rodents, but insufficient evidence existed to conclusively show toxicity in humans {NTP, 2002a). 
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HHEs and report back summary findings to NTP. Infrequently, NTP supports HHE ac- 
tivities, for example, an HHE related to cellulose insulation (Buchcr, 2008; NIOSH, 
200 Id). 

The HHE Program documented numerous productive arrangements with NIOSH 
laboratories for analytic purposes (NIOSH, 2007b). 



Finding 3.5: The HHE Program has a significant impact on the training, devel- 
opment, and placement of program alumni inside and outside the HHE Program 
in the occupational health community. 

A key output of the HHE Program is the occupational health professionals who have 
gone through training with the HHE Program. The HHE Program trains and places a 
large number of these professionals, constituting a significant transfer of human capital to 
the occupational health community through hosting international visitors, providing onc- 
to three-month rotations or joint residency programs for occupational medicine residents, 
offering "tag-along" opportunities to participate in HHEs, supporting positions for EIS 
officers, and supporting summer positions for minority students to learn about occupa- 
tional health (NIOSH, 2008e). HHE-trained professionals bring the skills, methodologies 
and habits developed by the HHE Program to a broad array of public and private agen- 
cies. The committee finds that the HHE Program plays a significant role in the training of 
the occupational health labor force through its participation in such programs as the 
Commissioned Officer Student Training and Extern Program (COSTEP) and EIS. The 
committee is impressed by the commitment to occupational health fostered by participa- 
tion in HHE Programs over the decades, attested to by committee members themselves 
and reflected in numerous written and oral statements made to the committee. 

The occupational health community would benefit from continued HHE Program en- 
gagement in strategic recruitment of trainees, perhaps by partnering more extensively 
with ERCs and universities. Committee members conclude, based on their own experi- 
ences, that universities could and should be doing more to recruit the occupational health 
labor force of the future, and that the HHE Program could play a substantial role in the 
training and placement of that labor force. The funding of these programs is largely out- 
side HHE Program control, which increases the need for the HHE Program to recruit ac- 
tively from the CDC program training ranks- 
There is limited tracking of program alumni (NIOSH, 2008e) and the committee finds 
little evidence that program alumni are used in any substantive capacity in advisory, case 
finding, or investigative roles (despite a high degree of enthusiasm and loyalty exhibited 
among program alumni encountered by the committee). The committee recommends the 
development of a program-level advisory board that would assist the HHE Program in 
leveraging its resources. In addition to program alumni, board members could include 
members of community groups and occupations not currently well served by the HHE 
Program. The advisory board could serve a recruiting and retention function, provide op- 
portunities to use program alumni, assist with ease finding, and provide expert advice 
about meeting some of the challenges facing the HHE Program given its limited re- 
sources and emerging challenges. 
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Recommendation: Use the HHE Program to develop occupational health profes- 
sional resources. This could be accomplished through 

a. Increased recruitment of new investigators from universities, EIS, COSTEP, 
occupational medicine residencies, ERCs, and state and local health depart- 
ments into HHE Program training rotations. This will require ongoing de- 
velopment of more attractive training, mentoring, and rotations. 

b. Tracking and mobilizing the extensive talent and commitment represented in 
the HHE Program-trained occupational health workforce. A network of 
HHE Program alumni could be fostered to help develop HHE opportunities. 
A program-level advisory board could assist the program in leveraging re- 
sources, serve a recruiting and retention function, assist in identifying emerg- 
ing issues, and provide expert advice. 

c. Engagement and use of ERCs and other university-based training programs 
to involve trainees in HHE field investigations. 

d. More formal collaborations with ERC faculty and other extramural re- 
searchers to assist with field investigation, dissemination, and training oppor- 
tunities. 

e. Enlisting the assistance of training program participants and alumni to limit 
the impact of program emergency response activities on routine program op- 
erations. 



4. TRANSFER OF PROGRAM-GENERATED HAZARD AND PREVENTION 
INFORMATION TO THE OCCUPATIONAL HEALTH COMMUNITY 

The committee is charged with examining how the HHE Program transfers program- 
generated information to the occupational health community, with a particular focus on 
whether the information is presented appropriately. In this section, the committee begins 
by defining the occupational health community as a heterogeneous group of people re- 
quiring different strategies for effective transfer. The committee then turns to an evalua- 
tion of the specific knowledge transfer and exchange mechanisms the HHE Program uses 
to reach target audiences in the occupational health community. 

The occupational health community is heterogeneous, and includes scientists, occupa- 
tional public-health professionals, the public-health labor force as a whole, companies, 
managers, members of the labor force, and institutions that represent individuals and 
groups. A comprehensive and appropriate technology transfer program would be targeted 
to the needs of this heterogeneous community. In order to assess this aspect of the HHE 
Program, the committee collected numerous examples of HUE communication strategies, 
solicited written comments from stakeholders, heard testimony from employers and em- 
ployees affected by the HHE Program, and received detailed responses to related ques- 
tions from HHE Program staff. Furthermore, a majority of the HHE Program perform- 
ance measures {listed in Table 3-1) arc directly related to transfer activities, including in- 
creased targeted marketing; increased number of links to the HHE Program website from 
external websites; greater use of cNcws and Epi-X; publishing of compendia of findings, 
annual reports, and peer-reviewed publications; participation in interagency activities; 
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and participation at trade meetings. Finally, recommendations related to transfer activities 
in prior evaluations of the HHE Program (RTl, 1997; NIOSH BSC, 1997, 2006) have 
been addressed by the HHE Program and are now represented in its current portfolio of 
knowledge transfer and exchange activities. 



Examples of Transfer to the Occupational Health Community 

Transfer Maierial Type: Numbered HHE reports 

Finding 4.1: Numbered HHE reports are generally of high quality, of direct im- 
mediate benefit to investigated sites, and of benefit to the larger occupational 
health community. 

As discussed in earlier chapters, the numbered HHE report is the primary output of 
the HHE Program, emanating directly from the enabling legislation and regulation. The 
committee reviewed numerous examples of numbered HHE reports, received input from 
report recipients, and solicited comments from the occupational health community about 
HHE reports. The committee also examined response patterns from the HHE foUowbaek 
survey program designed to assess satisfaction before, during, and after the conduct of an 
HHE investigation. HHE reports are generally of very high quality and useful to the spe- 
cific site investigated, and to the larger community of educated and informed occupa- 
tional health professionals. The plain-language summary "Highlights" page included at 
the beginning of every numbered HHE report was specifically mentioned by stakeholders 
as being of particular use to workers. 

Ongoing efforts in the development of the format of numbered HHE reports seem to 
increase both efficiency and effectiveness. When necessary, given the composition of the 
labor force, HHE reports are translated into other languages. The HHE Program is at its 
heart an applied research program; however, the fact that some HHE reports have been 
translated to the peer-reviewed scientific literature is testimony to their high quality. In its 
request for input, the committee heard very positive feedback about the quality, effi- 
ciency, and usability of the HHE report process and products. 



Transfer Material Type: Technical Assistance Letters 

Finding 4.2: Technical assistance letters appear to be an efficient use of HHE 
Program resources when problems are well understood. Because they are not 
disseminated widely, they are of little benefit to the larger occupational health 
community. 

As discussed in Chapter 3, the committee reviewed 10 examples of technical assis- 
tance letters written by the HHE Program staff to individual requestors in response to 
valid HHE requests. HHE Program staff indicated that technical assistance letters were 
generated when a considerable amount of expertise on a topic already exists, enabling 
staff to provide detailed information to the requestors. For example, a letter response to a 
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request concerning ventilation conditions in a health facility used the findings of a site- 
specific building inspection, a health inspection by the state authority, findings of a site 
safety committee, a report by the Institute of Medieine, a referral to relevant guidelines 
generated by a different state agency, national engineering standards, and a peer- 
reviewed publieation (NIOSH, 2007e). This four-page letter provided very detailed guid- 
anee for remedying the problem, with appropriate resources provided as enclosures. Ad- 
ditional examples of technieal assistance letters reviewed by the committee were on the 
topics of ventilation, exposures to well-defined chemical hazards, and exposure to bio- 
logical hazards. In general, the letters were well documented. One of the letter reports 
pertaining to biological hazards included seven references to the peer-reviewed and regu- 
latory literature (NIOSH, 2006e). Inclusion of such references in these reports effectively 
transfers information regarding sources of relevant information. These letters do not ap- 
pear to be disseminated widely, limiting the potential impact of the transfer of knowl- 
edge. 



Transfer Material Type: Peer-Reviewed Publications 

Finding 4.3: The HHE Program engages in a great deal of formal scientific pub- 
lication related to its research. 

The HHE Program provided the committee with 58 pages of titles of various papers 
disseminating HHE Program results dated from the early 1990s to the present. The titles 
include numbered HHE reports, journal articles, book chapters, NIOSH numbered docu- 
ments, and various other items appearing in trade journals. 

From 1996 to 2007, the HHE Program published well over 100 peer-reviewed publi- 
cations, including those in major medical journals, such as the Journal of the American 
Medical Association, the New England Journal of Medicine, the American Journal of In- 
dustrial Medicine, the Journal of Occupational and Environmental Medicine, and the 
American Journal of Surgical Pathology. HHE Program scientists have also authored 
peer-reviewed articles in the leading general journal of public health, the American Jour- 
nal of Public Health. Articles arc similarly well placed in toxicology {Neurological Toxi- 
cology, Toxicologist, Journal of Toxicology and Environmental Health, Critical Reviews 
in Toxicology); occupational health {American Industrial Hygiene Association Journal, 
Applied Occupational Environmental Hygiene. Journal of Occupational and Environ- 
mental Hygiene): and environmental health (Journal of Environmental Monitoring, Envi- 
ronmental Health Perspectives), among others. It is particularly noteworthy that HHE 
Program scientists have published 28 articles in the Morbidity and Mortality Weekly Re- 
port (MMWR). Though MMWR is not peer reviewed, it has been a highly visible and 
respected publication in public health for many years (NIOSH, 2007b). These publica- 
tions served to document advances in our knowledge of specific occupational illnesses 
and have had impacts on the development of treatment and intervention techniques to 
prevent or ameliorate the incidence of these illnesses. 

Program transfer activities can be very thorough in some areas, and transfer occurs in 
a variety of ways. Since 1990, for example, there have been 39 numbered HHE reports, 
20 journal articles, and 3 NIOSH numbered documents published — including proceed- 



Prepublication Version - Subject to Further Editorial Revision 



IMPACTS OF THE HEALTH HAZARD EVALUATION} PROGRAM -50 



ings of a workshop and a manual of analytical methods — all related to tuberculosis 
(NIOSH, 2007b). Additionally, seven items, such as chapters in books, and two items re- 
lated to OSHA rulemaking on occupational exposure to tuberculosis have been published. 
A more recent example is related to flavorings. Since 2002, 6 HHEs, 10 journal articles 
(including in the New England Journal of Medicine), and 1 NIOSH numbered document 
have been published. 



Transfer Materia! Type: Academic Presentations 

Finding 4.4: The HHE Program presents research flndings and staffs informa- 
tional booths at major academic conferences related to occupational health. 

The committee reviewed an extensive 10-year record of regular presentations at ma- 
jor academic conferences, many of which were later published in the peer-reviewed lit- 
erature. In addition to their substantial activities in the national occupational health com- 
munity, HHE Program scientists have presented research at international conferences in 
Austria, Brazil, Canada, China, Finland, Germany, Italy, Mexico, Scotland, Sweden, 
South Africa, and South Korea. Examples of domains and specific academic-focused 
conferences include the following 

• General: the American Industrial Hygiene Conference and Exposition, the Ameri- 
can Occupational Health Conference, the World Congress on Safety and Health at 
Work, and the American Statistical Association 

• Medical: American Thoracic Society, American Academy of Allergy, Asthma 
and Immunology, Society of Diagnostic Medical Sonography 

These presentations have been an effective mechanism for transfer of information from 
the NIOSH HHE Program to the public health community. The information communi- 
cated in the presentations can be expected to impact occupational health in a positive 
manner. 



Transfer Material Type: Trade Presentations 

Finding 4.5: There is evidence that HHE publishes in trade journals and makes 
presentations to trade association meetings, but more can be done to reach a 
wider audience and a greater diversity of employer and industry types. 

The committee reviewed a list of many presentations at trade conferences (NIOSH, 
2007b). Examples of such presentation audiences include those 

• in manufacturing: the American Automobile Manufacturers Association, the Na- 
tional Cotton Council of America, and the International Roofing Exposition 

• at professional associations: the National Hearing Conservation Association; and 
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• at regional meetings: the Kentucky-Tennessee Water Environment Association, 
the Phoenix Roofing Industry Silica Meeting, and the New England Biological 
Safety Association. 

Appearances at such venues are important and useful. However, the committee re- 
ceived stakeholder input from invited speakers and in response to the committee's written 
request for input that the HHE Program may not be presenting at a wide enough range of 
trade- and business-related venues. In particular, it was noted that the HHE Program 
could do more to reach out to small business associations. Furthermore, the characteris- 
tics of the construction industry and agricultural labor forces may require special outreach 
efforts. The associate director for Safety and Health Research at CPWR, noted a need to 
target small, high-risk residential construction companies that do not have the resources 
to investigate and solve their work-related safety and health problems {Gittlcman, 2008). 
It was noted that HHE Program success stories include lead and sihca. but the speaker 
cautioned that the mobile and contingent nature of the construction industry may result in 
worker exposure to well-understood risks. The speaker suggested that additional venues 
likely to provide good opportunity for dissemination of program information are the an- 
nual NIOSH Build Safe Conference or the Chicago Safety Council Conference. 

As described earlier in this chapter, the vice president of manufacturing for a small 
flavoring and fragrance company in California responded to questions during a commit- 
tee meeting discussion {Speakman, 2008). Discussion led the committee to conclude that 
presentations made at trade association meetings are one important mechanism to transfer 
information, but many small businesses do not join their trade association or participate 
in association activities. Furthermore, the social dynamic at trade association meetings 
may make it difficult for business owners to have open dialog about problems experi- 
enced with their production processes. 



Transfer Activity Type: Interagency Cooperation 

Finding 4.6: The HHE Program participates in a variety of intergovernmental 
activities that promote knowledge about occupational health matters. 

The committee finds substantial evidence of engagement by the HHE Program with 
some parts of the federal government. Examples include involvement — sometimes over 
long periods — with single or ongoing working groups, such as the EPA Biosolids Expo- 
sure Measurement Workshop, the OSHA/NIOSH Interagency Health Outcomes, the 
Navy Occupational Health and Preventive Medicine Workshop, and the NORA Industry 
Sector Meeting. 

Most of such efforts appear targeted at the federal level. As discussed in Chapter 3, 
the committee heard from some state and local health officials who were unaware of 
HHE Program activities, including those occurring within their own jurisdictions. One 
email respondent to the committee's request for input noted that the HHE Program was 
perhaps the "best kept secret in occupational safety and health" {Kicfer, 2007). A particu- 
larly poignant example of the program's "best-kept secret" status came from the commis- 
sioner of health of a major East Coast city. This individual had not heard of the HHE 
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Program prior to being invited to speak to ihc committee, in spite of his former employ- 
ment on tlie staff of U.S. Representative Henry Waxman (CA), wlio is known for his ac- 
tivities related to health and healthcare reform. The idea was reinforced in a majority of 
the responses to the committee's written request for input (see Appendixes C and D). Al- 
though many respondents were quite familiar with the HHE Program, the most common 
suggestion for improvement was for the program to increase its outreach and visibility so 
that its services would be extended. HHE Program staff responded that the program has 
regular contact with the Council of State and Territorial Epidemiologists; the Epidemic 
Intelligence Service; state and federal OSHAs; and other parts of NIOSH and CDC 
(NIOSH, 2007i;response to question #21). The response further noted, however, that 
these efforts have not been formally evaluated. To the extent that this committee has con- 
ducted its own evaluation, it seems that the HHE Program has had significant involve- 
ment with some parts of the federal government and with some states. The evidence sug- 
gests that the HHE Program could enhance these efforts to encompass all potential fed- 
eral and state partners. Furthermore, additional attention to local governmental agencies 
is also warranted. 



Transfer Material: Compendia of Known Findings 

Finding 4.7: The HHE Program has made progress in developing compendia of 
findings and recommendations about well-understood occupational health risks. 

The HHE Program has developed four compendia of findings about occupational 
health risks associated with isocyanates, noise, tuberculosis, and lead (NIOSH, 2004e, 
1998c, 2001b, and 2001a. respectively). The program is currently working on compendia 
on health risks faced by firefighters and those in the healthcare sector. 

The HHE Program tabulated the type and frequency of HHE reeommendafions, in- 
cluding 82 field investigafions between 2000 and 2005 for which followback surveys 
were conducted (NIOSH, 2007g:Table G [a]). Of the reports, 65 percent recommended 
engineering controls, 33 percent recommended exposure monitoring, and 28 percent sug- 
gested medical surveillance. Although HHEs are often thought of as hazard-specific ap- 
proaches to occupational health, the distribution of recommendafion types suggests that 
compilation of general process guidelines for use by employers, such as those already 
written for hazards including lead and tuberculosis (NIOSH, 2001a, b), may be an effi- 
cient use of program resources. For example, over half of HHE reports recommended 
administrative controls (79 percent), and a large number of reports also recommended 
housekeeping controls (44 percent), labor-management eommunicafion (28 percent), and 
use of personal protective equipment (27 percent). If generalized process-oriented find- 
ings can be gleaned from the experience of the HHE Program across a variety of settings, 
then the program may consider compiling compendia of such findings. 

Recommendation: Increase efforts to compile compendia of findings (such as those 
developed for isocyanates, noise, tuberculosis, and lead) when generalized process- 
oriented findings can be gleaned from the experience of the HHE Program over a 
variety of workplace settings. 
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Dissemination Strategies 

Finding 4.8: A wide variety of dissemination mechanisms, including the Internet, 
are targeted to the professional occupational health workforce. 

The committee has already noted significant program strengths in dissemination 
through peer-reviewed scientific literature and via academic, professional, and trade as- 
sociation meetings. This section focuses on the ability of the HHE Program to dissemi- 
nate findings to occupational health stakeholders beyond the professional labor force. The 
primary dissemination mechanism for HHE reports and other pubhshed materials is by 
email and the Internet. Reports are posted on the HHE Program website and Epi-X, and 
are announced in eNews, a NIOSH email newsletter. They are sent to, and made avail- 
able through, the National Technical Informafion Service and federal OSHA regional of- 
fices. All these mechanisms require (1) knowledge of the program and (2) a sophisticated 
ability to conduct an electronic search. The issue of whether people know about the HHE 
Program has already been discussed in detail. This section addresses the issue of an 
Internet-based disseminafion strategy. 

An Internet-based dissemination strategy does not serve people who do not have 
regular access to or knowledge about using the internet. Such populations may include 
non-native language speakers, the working poor, and ethnic and racial minorities. The 
committee also notes that the program's Internet-based dissemination mechanism is diffi- 
cult to maneuver, even for well-educated and well-informed healthcare professionals. 
Several respondents to the committee's request for input mentioned difficulty using the 
HHE Program website, and emphasized the difficulty in finding HHE reports (see Ap- 
pendix D for a summary of comments). Several written comments emphasized the diffi- 
culty of finding HHEs, and also addressed the need for improvements to the search en- 
gine. An illustrative comment from an anonymous occupational and environmental phy- 
sician and industrial hygienist in academe follows: 

The online search mechanism at the CDC website could be more user- 
friendly. In addition to the search by terms, an alphabetical categorization 
by main topic (exposure and/or disease) would be helpful. Also, the avail- 
ability of HHE data could be better publicized. I have been aware of HHEs 
for years, but rarely do I hear about them from other sources. 

Though the committee commends the HHE Program for making HHEs available 
online, committee members have had similar difficulty using the HHE search engine. In 
the preparation of Table 4-1, the committee noted apparent inconsistencies in the way 
keywords arc assigned to HHEs. For example, in an effort to locate HHEs related to 
work-related musculoskeletal disorders in the program's online search engine 
(http://www.cdc.gov/niosh/hhc/), 77 matches were found when a search was conducted 
using the search term "musculoskeletal hazards," 113 matches with "musculoskeletal," 
and 92 matches with "ergonomie." Approximately 63 percent of the "ergonomic" results 
appeared within the results of the search under ""musculoskeletal." Similar results were 
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observed using search terms such as "indoor environmental quality," "lEQ," "indoor air 
quality," "respiratory indoor environmental quality," and "respiratory lEQ." 

As early as the 1997 evaluation of the HHE Program by the Board of Scientific 
Counselors (NIOSH BSC, 1997), there were recommendations to improve linkage to ex- 
tant occupational websites at OSHA, labor unions, and other interested stakeholders. 
Eleven years later, such measurable performance goals appear in the HHE Program stra- 
tegic plan. Several respondents to this committee's request for input also noted the need 
for better linkage between the HHE Program and OSHA and other NIOSH websites. One 
respondent suggested that linkage between the HHE Program and local health depart- 
ments, state occupational safety agencies, and industry associations could be improved, 
and an invited speaker to a committee meeting from OSHA Region 1 suggested that HHE 
Program success stories could be posted on the OSHA website (Kent, 2007). The com- 
mittee encourages the HHE Program to work toward its performance measures of in- 
creasing the percentage of relevant websites with links to the HHE Program website (see 
Table 3-1). 

The committee was told by HHE Program leadership that NIOSH is responsible for 
HHE webpages, editorial and graphics staff, and some administrative staff services. Al- 
though seeking economies of scale makes sense for a program as small as the HHE Pro- 
gram, it is also important that NIOSH provide adequate support to assist in the program 
mission. As revealed by its difficulty of use, it would seem that the HHE Program web- 
site is not a high priority for NIOSH. 

The HHE Program appears aware of the limitations of reliance on the Internet for dis- 
semination, as reflected in the program's strategic priorities and program descriptions of 
outreach activities. Respondents to the committee's request for input identified a number 
of ways in which dissemination could be expanded, especially to reach underservcd 
populations. The responses are summarized in Appendix D. 

In brief, the HHE Program appears aware of the need to diversify its social marketing 
efforts, and program staff informed the committee that it has engaged in pilot networking 
in Cincinnati, has conducted limited customer surveys, and is currently planning a tar- 
geted marketing campaign as part of its strategic plan. The recent employment of a full- 
time health communications expert will likely help move the program in the right direc- 
tion. Participation in the NIOSH r2p initiative^ may be promising, but depends greatly 
on the adequacy of funding and support for that effort, about which the committee col- 
lected no information. 

Recommendation: Improve the searchability of the online HHE search engine by 
developing a list of standardized key words (an alphabetized listing of hazard and 
disease would be beneficial). 



'" Research to Practice (r2p) is a NIOSH initiative focused on (he transfer and translation of research find- 
ings, technologies, and information into highly effective prevention practices and products adopted in the 
workplace (http://\\"ww. cdc.gov/niosh/r2p/, accessed October 10, 2008), 
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Finding 5,1: The HHE Program is a recognized federal leader in its ability to re- 
spond quickly and effectively to new and emerging hazards, particularly in 
emergencies. 

As is the case for other aetivities of the HHE Program, measuring the impaet of HHE 
Program emergeney response activities is difficult. Data related to impact do not exist, 
and impact in this area is more difficult to assess because the HHE Program does not 
serve as the lead agency during emergency response. HHE staff often work in concert 
with staff from other agencies, and often under unusual circumstances, making contribu- 
tions by the HHE Program difficult to quantify. However, anecdotal evidence exists that 
can inform about the contributions of the program. As described in Chapters 2 and 3, 
HHE Program staff members are uniquely qualified to respond to emergencies, and arc 
often asked to assume leadership roles during emergency response. An important impact 
of the HHE Program, especially during emergency response, has been the subjective and 
effective validation of worker experiences and concerns. HHE Program procedures are 
established that allow staff to listen to workers in order to effect change and improve 
conditions. 

Some noteworthy examples of the HHE Program's recognized excellence in emer- 
gency response arc the prominent roles staff have served in response to the World Trade 
Center disaster on September 11, 2001 (9/11). HHE staff participated in etiologic investi- 
gations of occupational exposures and the health effects among workers in a building 
close to the disaster site three months after 9/1 1 {Trout et al., 2002), and in later investi- 
gations of stress-related symptoms throughout the city (MMWR, 2002). The program 
also made important contributions in response to the 2001 anthrax attacks made through 
the U.S. Postal Service (USPS), highlighted in Box 4-1, and responding to issues related 
to the health effects of handling and opening of mail irradiated to defend against biohaz- 
ards (NIOSH, 2002c, c). 

In response to natural disasters, the HHE Program conducted investigations of the 
mental and physical health of the New Orleans Police and Fire Departments following 
Hurricane Katrina (NIOSH, 2006c, d). As part of a CDC team, the HHE Program assisted 
local, state, and federal agencies in addressing occupational safety and health issues, per- 
forming surveillance and exposure assessments among workers, performing outreach to 
vulnerable workers, and disseminating occupational health data as the State of Louisiana 
and City of New Orleans rebuilt the city's public-health system. The HHE Program also 
provided international assistance in the aftermath of the 2004 Indonesian tsunami that 
killed more than 225,000 people. HHE investigators coordinated and prioritized dona- 
tions to speed the selection and manufacturing of personal protective and other equipment 
(NIOSH, 2007b). 

The committee regards the information revealed through investigations conducted 
during the course of emergency response as useiul and believes it will likely help re- 
spondcrs in the event of future similar emergencies. Some emergencies, such as future 
anthrax or other biohazard attacks, may be avoided or their damaging effects mitigated, 
because of the guidelines and definitive recommendations established by NIOSH and the 
HHE Program. The U.S. Government Accountability Office (GAO) submitted a report to 
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BOX 4-1 
HHE Program Response to Anthrax Contaminations 

In October 2001, Bacillus anthracis (anthrax) spores were received in the mail by a news 
media office in Florida. HHE staff members were part of the response team that conducted the 
environmental evaluations of the affected building and of USPS offices where the mail was proc- 
essed (NIOSH, 2007b). Ultinnately, HHE investigators participated in 26 more emergency re- 
sponse investigations involving potential anthrax contamination in Washington, DC, Florida, Con- 
necticut, New Jersey, New York, Missouri, and Texas in a sl^ort period of time. Fifty-five separate 
sites were evaluated in New Jersey alone. HHE staff collaborated with researchers inside and 
outside NIOSH to design reliable standardized protocols for collecting samples and to use exist- 
ing sampling technologies in innovative ways. Onsite technical assistance was delivered as part 
of the initial response to suspected biological threats (NIOSH, 20Q2b). NIOSH worl<ed with other 
CDC offices, OSHA, and USPS to develop and disseminate guidelines on engineering controls 
and personal protective equipment for postal workers, mail handlers, and first responders (CDC, 
2001; NIOSH, 2001c, 2002c), and helped design clean-up adequacy standards, implement de- 
contamination procedures, and conduct post-clean-up assessments. NIOSH also trained Federal 
Bureau of Investigation and U.S. Coast Guard personnel, independent contractors, and others in 
appropriate anthrax decontamination procedures. 

Information transfer was conducted by a variety of means. The emergency response teams 
communicated directly with teams at the investigated sites and helped staff the CDC command 
center, which provided 24-hour assistance for workers, employers, and the general public for an 
extended period of lime. It is presumed that information disseminated through these mechanisms 
was based largely on the findings and expertise of HHE staff participating in the investigations. 
Additionally, HHE investigators authored or co-authored 17 peer-reviewed publications on the 
topic of anthrax; made 38 technical presentafions; published 1 numbered HHE report and 3 web- 
based guidance documents; and testified to a congressional subcommittee regarding microbi- 
ological sampling methods for anthrax (NIOSH, 2007b). Within CDC, an educational video was 
produced for the benefit of postal workers (CDC, 2002). 

Development of training tools and research in anthrax detection methodologies continued, 
stemming from the results of initial investigations. OSHA developed an illustrated and interactive 
web-based training tool on anthrax (OSHA. 2003). Sandia National Laboratories collaborated with 
Lawrence Livermore National Laboratories to develop a tool to assist clean-up personnel and 
officials at airports, other transportation centers, and high-traffic public buildings in the reoccupa- 
tion of buildings following biological contamination, and tested the tool with the assistance of 
NIOSH staff (Sandia National Laboratories, 2005). The research for this tool was sponsored by 
the Department of Homeland Security and included partnerships with the San Francisco Bay area 
airports. 

Reoccupation of facilities contaminated with anthrax and remediated and tested by invesfiga- 
live teams resulted in no new cases of the disease, indicating successful remediation efforts. 



the House Subcommittee on National Security, Emerging Threats, and International Rela- 
tions on agencies' need to validate sampling activities in order to increase the confidence 
in negative sampling results (GAO, 20fl5a). Although this report examined USPS, CDC, 
and EPA sampling methodologies, NIOSH provided guidance to USPS during its sam- 
pling and was consulted by GAO during preparation of the report. 

Other NIOSH programs may have been informed by HHE Program activities, as 
exhibited by sueh publications as Guidance for FUlraiion amJ Air-Cleaning Systems to 
Protect Building Environments from Airborne Chemical, Biological, or Radiological 
Attacks (NIOSH, 2003e), and the NIOSH Interim Recommendations for the Cleaning and 
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Remediation Flood-contaminated HVAC Systems: A Guide for Building Owners and 
Managers (NIOSH, 2005d). The NIOSH National Personal Protective Technology 
Laboratory contracted with the RAND Science and Technology Policy Institute to review 
available databases related to disease, injury, and fatality data associated with personal 
protective technologies in emergency response (Houser et al., 2004). 

Any evaluation of the impact of the HHE Program should consider that the program's 
resources (for example, funding, staff, scientific capacity) can and will be commandeered 
during a national emergency. The committee is not able to examine the effect of emer- 
gency response activities on day-to-day program activities in great detail because of the 
non-routine nature of emergency response. Financial repercussions of individual re- 
sponses may differ depending on how the program is compensated for response activities. 
While emergency response may result in important positive impacts by those affected by 
the emergency, the program and NIOSH should remain diligent to avoid negative impact 
on the protection of human health in more routine occupational settings. 

Recommendation: The HHE Program and NIOSH should remain diligent to avoid 
negative impact on the routine activities of the HHE Program as a result of emer- 
gency response activities. 



IMPACT SCORE 

As noted in the initial part of this chapter, it is difficult to find quantitative data to 
support an estimation of the degree of impact of the HHE Program in terms of decreases 
in numbers of workplace illnesses. Yet there is evidence that workers in investigated or 
similar workplaces have experienced reductions in exposures to health hazards and inap- 
propriate work practices as a result of activities of the HHE Program. Because these re- 
ductions would be expected to lead to improved health for the workers, the scoring of 
impact of the HHE Program by the committee is based on intermediate outcomes, namely 
reductions in exposures to health hazards or an improvement in work practices. 

The consensus of llie committee is that the HHE Program can be highly effective in 
investigating hazards and providing advice to workplaces that make requests. The HHE 
Program is unique in the occupational health community in investigating unanticipated or 
underappreciated hazards in the workplace and relating them to the exposure or occupa- 
tional circumstances of the worker. However, the committee is concerned that the pro- 
gram is not reaching all occupational groups, particularly employees of small businesses 
and underscrved populations. There is also a concern that limited funding and obligations 
for emergency responses might dilute the effectiveness of the program and cause it to 
veer off into doing more routine industrial hygiene reviews. 

The transfer of information to workplaces other than those mvestigatcd and to other 
agencies in the occupational health community has been extensive; however, the transfer 
is incomplete in that many people interviewed by the committee were unaware of the 
HHE Program. Therefore, more emphasis needs to be placed on reaching out to the whole 
occupational health community. 

The HHE Program is found to have a strong impact on the development of NIOSH- 
reeommcndcd occupational guidelines and on the support of NIOSH testimony for ^ro- 
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posed OSHA rules. The HHE Program also contributes significantly to the advancement 
of scientific knowledge through its pubUcations in the open literature and interactions 
with research programs within NIOSH and other governmental agencies. Training pro- 
grams through the HHE Program were thought to offer an excellent opportunity for in- 
creasing expertise in occupational health, and such programs, as well as the use of alumni 
from these programs in emergencies, should be expanded. 

Based on the scoring system developed by the Framework Committee (see Chapter 1, 
Box 1-1) the committee ranks the impact of the HHE Program as 4. If the committee had 
not been restricted to using integer scores, it would have scored the impact of the HHE 
Program as between a 4 and a 5. 
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Vision and Recommendations 



In previous chapters, the committee evaluated the relevance and impact of the 
Health Hazard Evaluation (HHE) Program. In this chapter, the committee de- 
scribes its vision of what the HHE Program should be. The vision deseription 
provides a picture of what the ideal program would look like, including elements 
of the existing HHE Program. The committee concludes the chapter by summariz- 
ing recommendations provided in Chapters 3 and 4 of the report. 



V[SION FOR THE HEALTH HAZARD EVALUATION PROGRAM 

The ideal HHE Program, as envisioned by the committee, would promptly 
respond to requests for evaluation of the chemical, physical, and biological haz- 
ards within the HHE legislative scope, serving all workers, including underserved 
populations. The program would do what the National Institute for Occupational 
Safety and Health (NIOSH) can do uniquely — emphasize identification of health 
effects by combining medical investigation and industrial hygiene (exposure as- 
sessment) techniques with epidemiological and clinical toxieologieal perspectives, 
which may also involve development of new measurement and control tech- 
niques. The program would communicate results promptly to the target work- 
place, to workplaces with common exposures, and more broadly to the public 
health community. In addition, the program would respond to requests from clini- 
cians and community organizations with ties to workers, in addition to the current 
legislatively supported requests. After completion of investigations, staff would 
follow back on individual investigations to document and report abatement of 
hazards as well as follow back on grouped investigations to document abatement 
on a national basis. The ideal program would appeal to company and industry 
leaders, drawing them in as partners to prevent illness and injury industry-wide. 

Ideally, the HHE Program would be at the cutting edge of identification of 
emerging occupational health hazards. This could be done through systematic use 
of scientific literature and surveillance data, including that generated by NIOSH, 
and effective knowledge management. In this way, the HHE Program could iden- 
tify heretofore unrecognized hazards as well as known hazards for which permis- 
sible exposure limits (PELs) appear inadequate or for which an HHE would facili- 
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tate Occupational Safety and Health Administration (OSHA) standard setting 
when information is limited. The program would also work toward identifying 
new industries or workplaces where there arc limited data on potential hazards, 
and toward providing opportunities for application of other NIOSH resources, 
such as toxicology or control technology design responses to emerging workplace 
issues. 

HHEs would consider opportunities to confirm or deny health effects arising 
from occupational exposures. HHEs are opportunities to identify gaps in protec- 
tion at target workplaces, and indicators of possible problems at similar work- 
places generally. HHEs arc opportunities to describe problematic exposure cir- 
cumstances that may be used as teaching examples. The program would be used 
to train field investigators, including NIOSH employees and those from partner 
agencies, academic institutions, and from international counterparts. 

Finally, the HHE Program would continue to provide guidance and recom- 
mendations during public-health emergencies. 

In contrast, the HHE Program would not provide what others can do, sueh as 
routine industrial hygiene measurements, or standardized responses to common 
problems like indoor environmental quality. For such situations, the best use of 
HHE expertise would be to develop generalized protocols for response to com- 
mon problems. 



RECOMMENDATIONS 

Within current legislative funding constraints, the HHE Program has managed 
to perform well. The HHE Program is a highly relevant program that has had and 
promises to continue having a valuable impact on improving worker health. The 
committee recommends the program be continued and, as possible, expanded to 
conduct more field investigations over a greater range of hazards, especially 
among underservcd populations. In this section, the committee makes general 
recommendations for program improvement. The recommendations are organized 
into eight categories and presented in an order consistent with the HHE Program 
logic model. The chapter and page numbers where recommendations appear in the 
text are provided. The committee does not set priorities among its rceommenda- 
tions; it prefers encouraging implementation of any of the recommendations as 
resources are available. The committee recognizes that implementation of many 
of these recommendations will depend on the availability of resources, but, at the 
request of NIOSH, it refrains from making recommendations regarding resource 
allocations. The next step for the HHE Program is to consider how to allocate re- 
sources. 



Recommendation 1: Conduct regular assessments of performance meas- 
ures to determine whether available resources allow more ambitious 
goals (Chapter 3, page 40; see Table 3-! for recommendations related to spe- 
cific goals). 
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Recommendation 2: Improve the mechanisms by which requests for 
HHEs are sought and prioritized to include a broader array of requests 
from a wider variety of requestors. The program could achieve this through 

a. Systematic use of professional meetings, scientific conferences, scientific 
literature, and surveillance data, including those generated by NIOSH, to 
assist in prioritizing field investigations and recognizing emerging issues 
(Chapter 3. page 53). 

b. Implementing, as part of the triage process, a formal technical assistance 
mechanism to help requestors to formulate valid HHE requests. In cases 
where an HHE is not appropriate or where resource limitations prohibit an 
investigation, technical assistance should include referral to more appro- 
priate NIOSH divisions or government agencies (Chapter 3, page 63). 

e. Development of an explicit, written process for classifying and prioritizing 
HHE requests. Priority should be based on the gravity of the potential 
harm, the number of employees potentially at risk at similar workplaces or 
using similar work processes, the urgency of the problem, the potential to 
assess health outcomes, and the possibility of identifying emerging issues. 
Potential impact on standards and policy should also enhance the priority 
of an HHE request in the triage process. Relationship of the HHE to cur- 
rent research may be considered, but should not be the only or primary 
factor. The process should provide guidance on weighting these varying 
factors (Chapter 3, page 63). 

d. Better formalizing of the triage process, including the identification of 
needed expertise, and improving the transparency of the process to HHE 
requestors, while maintaining flexibility and speed (Chapter 3, page 63). 

e. Establishment of formal relationships with organizations representing un- 
derserved populations, small businesses, and their employees (Chapter 3. 
page 58). 

f Enhancing HHE Program outreach to OSHA national and regional offices 
and to state health and labor departments to better communicate the func- 
tions and activities of the HHE Program, increase cooperation with these 
agencies, and provide more complete and timely feedback (Chapter 3, 
page 57). 



Recommendation 3: Ensure that recommendations in HHEs are relevant, 
feasible, effective, and clearly explained. Such steps may include 

a. Explanation of the relevance, feasibility, and impact of each recommenda- 
tion in the text of HHE reports (Chapter 3. page 67). 

b. Priority-setting among recommendations in all reports to indicate those 
requiring immediate action in the targeted workplace (Chapter 3, page 

67). 
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c. Debriefing in NIOSH after site visits and report dissemination for deter- 
mination of relcvanee and impaet on a systematic basis {potentially missed 
opportunities to identify emerging health hazards could also be identified) 
(Chapter 3, page 67). 

d. Modification of the followbaek surveys for use in assessing the relevance, 
feasibility, and impact of recommendations {Chapter 3, page 68: Chapter 
4. page 81). 

c. Enhancement of internal quality assurance by development of a formal 
program. Consider external review of a sampling of recent reports and 
technical assistance letters for scientific content, report completeness, and 
appropriateness of recommendations (Chapter 3, page 68). 



Recommendation 4: Use the HHE Program to develop occupational 
health professional resources. This could be accomplished through 

a. Increased rccniitmcnl of new investigators from universities, the Epidemic 
Intelligence Service (ElS), the Commissioned Officer Student Training 
Extern Program (COSTEP), occupational medicine residencies, Education 
and Research Centers for Occupational Safety and Health (ERCs), and 
state and local health departments into HHE Program training rotations. 
This will require ongoing development of more attractive training, mentor- 
ing, and rotations (Chapter 4, page 96). 

b. Tracking and mobilizing the extensive talent and commitment represented 
in the HHE Program-trained occupational health workforce. A network of 
HHE Program alumni could be fostered to help to develop HHE opportu- 
nities. A program-level advisory board could assist the program in lever- 
aging resources, serve a recruiting and retention function, assist in identi- 
fying emerging issues, and provide expert advice {Chapter 4, page 96). 

c. Engagement and use of ERCs and other university-based training pro- 
grams to involve trainees in HHE field investigations (Chapter 4. page 
96). 

d. More formal collaboration with ERC faculty and other extramural re- 
searchers to assist in field investigation, dissemination, and training oppor- 
tunities (Chapter 4, page 96). 



Recommendation 5: Develop a proactive, comprehensive information- 
transfer strategy for HHE Program outputs with better approaches to 
reaching wider audiences, including traditionally underserved popula- 
tions. The HHE Program could 

a. Use innovafive techniques to reach small businesses and underserved 
populafions, creating a broad array of mechanisms for communicating 
with diverse consfituencies and attending to issues of literacy, language, 
and national-origin barriers. The effectiveness of applied outreach should 
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be evaluated in a formal manner (Chapter 3. page 59; Chapter 4, page 
87). 

b. Improve the searchability of the online HHE search engine by developing 
a list of standardized key words {an alphabetized list of hazards and dis- 
eases would be beneficial). (Chapter 4. page JOS). 

c. Develop distribution mechanisms that arc not Internet-dependent to com- 
plement Internet distributions (Chapter 4, page 87). 

d. Disseminate HHE results more broadly to groups likely to be affected, in- 
cluding distribution of HHE reports in the geographic regions where in- 
vestigations are conducted (Chapter 4, page 87). 

c. Increase efforts to compile compendia of findings {such as those devel- 
oped for isocyanates, noise, tuberculosis, and lead) when generalized 
process-oriented findings can be gleaned from the experience of the HHE 
Program in a variety of settings (Chapter 4, pages 88 and 102). 

f. Develop improved methods of outreach to stakeholders so that workers 
and workplaces affected by new and emerging occupational health prob- 
lems will be alerted quickly (Chapter 4, page 88). 

g. Supplement program outreach efforts by using community and small- 
business groups to translate HHE results and findings for their constituen- 
cies (Chapter 4, page 88). 

h. Leverage existing NIOSH, Centers for Disease Control and Prevention 

(CDC), and Department of Health and Human Services {HHS) resources 

to enhance technology transfer (Chapter 4. page 88). 
i. Evaluate, in a formal manner, the effectiveness of information-transfer 

programs, including knowledge transfer to employers and employees not 

investigated (Chapter 4. page 88). 



Recommendation 6: Develop more e\tensive formal linkages and mecha- 
nisms with other parts of NIOSH, CDC, and HHS to enhance the capac- 
ity for involvement in policy-relevant impacts through 

a) Promotion and increase in direct communication, especially with OSHA 
and state occupational safety and health agencies (Chapter 4, page 91). 

b) Alerts to NIOSH and CDC about HHBs that arc relevant to policy-making 
outside the CDC system (Chapter 4, page 91). 

c) Continued regular use of the National Occupational Research Agenda 
{NORA) sector councils and the NIOSH Board of Scientific Counselors to 
disseminate information about the HHE Program (Chapter 4. page 91). 

d) Pursuit of a change in the HHE Program's legislative and regulatory au- 
thority to improve the capacity to identify hazards in need of HHEs, im- 
prove the ability to gain entrance to facilities when requested by treating- 
physicians or community representatives, and address exposures other 
than chemical agents (Chapter 4, page 91). 
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Recommendation 7: Initiate formal periodic assessment of new and 
emerging hazards. To accomplish this, the HHE Program could 

a. Evolve from a program that passively receives requests to a proactive pro- 
gram that seeks opportunities for field investigations (Chapter 3, page 71). 

b. Develop systematic approaches to identify hazards where OSHA permis- 
sible exposure limits are inadequate or nonexistent, to identify unknown 
hazards, and to identify known hazards encountered under new circum- 
stances (Chapter 3, page 71). 

c. Establish and periodically review a tickler file of inconclusive or unex- 
pected evaluation results to determine whether new trends or problems 
may be emerging (Chapter 3, page 71). 

d. Periodically meet with intramural and extramural research scientists and 
stakeholders in government, academe, labor, and industry to discuss spe- 
cific unresolved evaluations, to review aggregate findings, and to solicit 
input about new or emerging hazards or interventions. The HHE Program 
could establish one or more stakeholder groups to assist in identifying ex- 
posure circumstances or types of workplaces that could be the object of 
HHE requests likely to have high relevance and impact. The NORA sector 
councils may serve this function (Chapter 3, page 71). 



Recommendation 8: Continue to provide guidance and recommendations 
during public health emergencies. To accomplish this, the HHE Program 
could 

a. Work with NIOSH management to remain diligent to avoid negative im- 
pact on routine activities of the HHE Program as a result of emergency re- 
sponse activities (Chapter 4, page 106). 

b. Develop a mechanism, such as the enlistment of help from training pro- 
gram participants and alumni, to ensure continuation of routine operations 
in the absence of staff involved in emergency response (Chapter 4, page 
112). 
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Framework for the Review of Research 

Programs of the National Institute for 

Occupational Safety and Health 

Version 8/10/2007 



This is the second version of a document prepared by the National Academies 
Committee for the Review of NIOSH Research Programs,' also referred to as the 
Framework Committee. This document is not a formal report of the National 
Academies — rather, it is a framework proposed for use by multiple National 
Academies evaluation committees to review up to 15 National Institute for Occu- 
pational Safety and Health (NIOSH) research programs. It is a working document 
subject to modification by the Framework Committee on the basis of responses 
received from evaluation-committee members, NIOSH, stakeholders, and the 
general public during the course of the assessments. 

This version rcfiects several significant changes to the original framework 
document (version 12/19/05) that was used to guide the work of the first four 
evaluation committees {Hearing Loss; Mining; Agriculture. Forestry, and Fishing; 
and Respiratory Disease). Changes were made in response to feedback from 
members and staff of these committees, as well as other comments on the original 
framework, in order to make the document more useful to evaluation committees 
as they carry out their work, (n particular, the following changes were made to the 
framework document during the revision process: 



Members oTlhc commillee ul ihe lime ihis veision was proiJuceJ were David Wcgman. chair (Unwersiiy 
of Massachuselts Lowell School of Health anil Cnviiorimeni), William Bunn III (Inlemaiional Truck and 
Engine Corporalion), Carlos Camargo (Haivard Mediual School), Susan Co/zcns (Georgia InsliUile ofTech- 
nology), Lctiiia Davis (Massachusctis Deparimenl of Public Health), James Deariiig (Kaiser Permunenle- 
Colorado), Fred Mclller Jr. (University of New Mexico School ofMediciiie), Franklin Mirer (Hiinler College 
School of lleahh Sciences), Jacqueline Nowell (United Food and ('omniercial Workers Inlemational Union). 
Raja Ramani (Pennsylvania Sluie University), Jornia Raiilanen (Inlemjlional Commission on Occupational 
Health), Rosemary Sokas (llniver=.ily of Illinois at Chicago School of Public Health), Richard Tucker 
(Tucker and Tucker Consultants, Inc. and University ofTexas at Auslinl, and James Zuiches (North Carolina 
Slate University). Sammantha Magsino (National Academies stafO was ihc study director. Joseph Wholey 
(Universily of Southern California), tbrmer commiUee member, contributed to ihc first version of this docu- 
nient. Part V includes brief biographies of current commillee members. 
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• the wording of some of the relevance and impact scores were edited to 
make the wording more precise and to reduce situations where the original 
scores were non-unique or overlapping (revised scoring criteria are given 
in Boxes 2 and 3); 

• a new tabic was added to provide explicit guidance to evaluation commit- 
tees on how to weigh differences in the observed levels of "research prior- 
ity" and "engagement in appropriate transfer activities" in arriving at a 
single integer score for relevance (see Table 6); 

• the guidance on scoring was clarified to make more explicit that all scores 
are to be given as integers; 

• the NIOSH logic model was updated (see Figure 1); 

• the tabic on evaluation committee information needs (Table 2) was re- 
organized to be more consistent with the NIOSH logic model, and addi- 
tional information needs identified by the first set of evaluation commit- 
tees were added; 

• a worksheet to assist with the development of scores has been deleted and 
key components of the worksheet have been incorporated into appropriate 
sections throughout the document; 

• the organization of the document was modified to more closely follow the 
revised statement of task and to improve readability; and 

• a number of sections of text originally presented in outline form were 
modified in tables or boxes to make the information more accessible. 

This second version of the framework document remains a working document sub- 
ject to further modification by the Framework Committee on the basis of input 
received from evaluation-committee members, NIOSH, stakeholders, and the 
general public during the course of the assessments. 
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Abbreviations and Acronyms 

ABLES Adult Blood Lead Epidemiology and Surveillance 

AOEC Association of Occupational and Environmental Clinics 

BLS Bureau of Labor Statistics 

CDC Centers for Disease Control and Prevention 

CSTE Council of State and Territorial Epidemiologists 

DOD US Department of Defense 

EC Evaluation Committee 
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FC Framework Committee 

HHE Health Hazard Evaluation 

MSHA Mine Safety and Health Administration 

NIH National Institutes of Health 
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OSHA Occupational Safety and Health Administration 
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TJVIT tools, methods, or technologies 

USDA US Department of Agriculture 
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1. INTRODUCTION 

In September 2004, the National Institute for Occupational Safety and Health 
(NIOSH) contracted with the National Academies to conduct a review of NIOSH 
research programs. The goal of this multiphase effort is to assist NIOSH in in- 
creasing the impact of its research efforts that arc aimed at reducing workplace 
illnesses and injuries and improving occupational safety and health. The National 
Academies assigned the task to the Division on Earth and Life Studies and the 
Institute of Medicine, 

The National Academies appointed a committee of 14 members, including per- 
sons with expertise in occupational medicine and health, industrial health and safety, 
industrial hygiene, epidemiology, civil and mining engineering, sociology, program 
evaluation, communication, and toxicology; representatives of industry and of the 
workforce; and a scientist experienced in international occupational-health issues. 
The Committee on the Review of NIOSH Research Programs, referred to as the 
Framework Committee (FC), prepared the first version of this document during meet- 
ings held on May 5-6, July 7-8, and August 15-16, 2005. This second version was 
finalized after the Framework Committee's May 30-31, 2007 meeting, based on 
feedback received on the framework from the first two independent evaluation com- 
mittees, NIOSH leadership, and National Academies' staff, as well as discussions 
during an earlier FC meeting in April 2006. 

This document is not a report of the National Academies; rather, it presents 
the evaluation framework developed by the FC to guide and provide common 
structure for the reviews of as many as 15 NIOSH programs during a 5-ycar pe- 
riod by independent evaluation committees (ECs) appointed by various divisions 
and boards of the National Academies. It is a working document to be shared with 
NIOSH and the public. This version has been modified by the FC on the basis of 
responses from the ECs, NIOSH, NIOSH stakeholders, and the public; and it may 
be modified again. It is incumbent on the ECs to consult with the FC if portions of 
the evaluation framework presented here are inappropriate for specific programs 
under review. 



I.A Overview of Charge to Evaluation Committees 

At the first meeting of the FC, Lewis Wade, NIOSH senior science adviser, 
emphasized that a review of a NIOSH program should focus on the program's 
relevance to and impact on health and safety in the workplace. In developing a 
framework, the FC considered the following elements of the charge to the ECs: 

1. Assessment of the program's contribution, through occupational safety 
and health research, to reductions in workplace hazardous exposures, ill- 
nesses, or injuries through 

a. An assessment of the relevance of the program's acfivities to the 
improvement of occupational safety and health. 

b. An evaluation of the impact that the program's research has had in 
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reducing work-related hazardous exposures, illnesses, and injuries. 
The evaluation committee will rate the performance of the program for its 
relevance and impact using an integer score of 1-5. Impact may be as- 
sessed directly (for example, on the basis of reductions in illnesses or inju- 
ries) or, as necessary, by using intermediate outcomes to estimate impact. 
Qualitative narrative evaluations should be included to explain the nu- 
merical ratings. 
2. Assessment of the program's effectiveness in targeting new research areas 
and identifying emerging issues in occupational safety and health most 
relevant to future improvements in workplace protection. The committee 
will provide a qualitative narrative assessment of the program's efforts and 
suggestions about emerging issues that the program should be prepared to 
address. 



I,B Evaluation Committees 

Individual ECs will be formed in accordance with the rules of the National 
Academies for the formation of balanced committees. Each EC will comprise per- 
sons with expertise appropriate for the specific NIOSH research program under 
review and may include representatives of stakeholder groups (such as labor un- 
ions and industry), experts in technology and knowledge transfer, and program 
evaluation. The EC will gather appropriate information from the sponsor (the 
NIOSH research program under review), stakeholders affected directly by NIOSH 
program research, and relevant independent parties. Each EC will consist of about 
10 members, will meet about three times, and will prepare a report. The National 
Academics will deliver the report to NIOSH within 9 months of the first meeting 
of the EC. EC reports are subject to the National Academies report-review proc- 
ess. 



I.C NIOSH Strategic Goals and Operational Plan 

As a prelude to understanding the NIOSH strategic goals and operational plan. 
NIOSH research efforts should be understood in the context of the Occupational 
Safety and Health Act (OSHAct), under which it was created. The OSHAct iden- 
tifies workplace safety and health as having high national priority and gives em- 
ployers the responsibility for controlling hazards and preventing workplace injury 
and illness. The act creates an organizational framework for doing that, assigning 
complementary roles and responsibilities to employers and employees, the Occu- 
pational Safety and Health Administration (OSHA), the states, the Occupational 
Safety and Health (OSH) Review Commission, and NIOSH. The act recognizes 
NIOSH 's role and responsibilities to be supportive and indirect, NIOSH research, 
training programs, criteria, and recommendations are intended to be used to in- 
form and assist those more directly responsible for hazard control (OSHAct Sec- 
tions 2b, 20, and 22). 
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Section 2b of the OSHAct describes 13 interdependent means of accom- 
plishing the national goal, one of which is "by providing for research . . . and by 
developing innovative methods . . . for dealing with occupational safety and 
health problems". Sections 20 and 22 give the responsibility for that research to 
NIOSH. NIOSH is also given related responsibilities, including the develop- 
ment of criteria to guide prevention of work-related injury or illness; develop- 
ment of regulations for reporting on employee exposures to harmful agents; 
establishment of medical examinations, programs, or tests to determine illness 
incidence and susceptibility; publication of a list of all known toxic substances; 
assessment of potential toxic effects or risks associated with workplace exposure 
in specific settings; and conduct of education programs for relevant profession- 
als to carry out the OSHAct purposes. NIOSH is also responsible for assisting 
the secretary of labor regarding education programs for employees and employ- 
ers in hazard recognition and control. 

The NIOSH mission is "to provide national and world leadership to prevent 
work-related illness, injury, disability, and death by gathering information, con- 
ducting scientific research, and translating the knowledge gained into products 
and services". To fulfill its mission, NIOSH has established the following strate- 
gic goals: 

• Goal 1: Conduct research to reduce work-related illnesses and inju- 
ries. 

o Track work-related hazards, exposures, illnesses, and injuries for pre- 
vention. 

o Generate new knowledge through intramural and extramural research 
programs. 

o Develop innovative solutions for difficult-to-solvc problems in high- 
risk industrial sectors. 

• Goal 2: Promote safe and healthy workplaces through interventions, 
recommendations, and capacity building. 

o Enhance the relevance and utility of recommendations and guidance. 
o Transfer research findings, technologies, and information into practice. 
o Build capacity to address traditional and emerging hazards. 

• Goal 3: Enhance global workplace safety and health through interna- 
tional collaborations. 

o Take a leadership role in developing a global network of occupational 
health centers. 

o Investigate alternative approaches to workplace illness and injury re- 
duction and provide technical assistance to put solutions in place. 

o Build global professional capacity to address workplace hazards 
through training, information sharing, and research experience. 



'Sec hllp://www cdc.gov/nio.sh/docs/.straiL'gic/. 
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In 1994, NIOSH embarked on a national partnership effort to identify re- 
search priorities to guide occupational health and safety research for the next 
decade. The National Occupational Research Agenda (NORA) identified 21 
high-priority research subjects (see Table 1). The NORA was intended not only 
for NIOSH but for the entire occupational health community. In the second dec- 
ade of the NORA, NIOSH is working with its partners to update the research 
agenda, using an approach based on industry sectors. NIOSH and its partners are 
working through sector research councils to establish sector-specific research 
goals and objectives. The emphasis is on moving research to practice in work- 
places through sector-based partnerships. 



TABLE 1 NORA High-Priority Research by Category 



Category 



Priority Research Area 



Disease and injury 



Allergic and irrilanl dennatitis 

Asthma and chronic obstructive pulmonary disease 

Fertility and pregnancy abnonnalities 

Hearing loss 

Infectious diseases 

Low-back disorders 

Musculoskeletal disorders of upper extremities 

Trauma 



Work environment and workforee 



Emerging technologies 
Indoor environment 
Mixed exposures 

Organization of work 
Special populations at risk 



Research tools and approaches 



Cancer research methods 

Control technology and personal protective 

equipment 
Exposure-assessment methods 
Health-services research 
Inlervenlion-effectiveness research 
Risk-assessment methods 

Social and economic consequences of workplace ill- 
ness and injury 
Surveillance research methods 



Figure 1 is the NIOSH operalional plan, presenled as a logic model, ■* of the path from 
inputs to outcomes for each NIOSH research program. The FC adapted the model to de- 
velop its framework. NIOSH will provide similar logic models appropriate to each re- 
search program evaluated by an EC. 



^Developed by NIOSH wilh ihe a.ssisliince ofihc RAND Corporaiioii, 
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Mission: To Provide National and World Leadership to Prevent Work-Related Illness and Injuries 




End 
Outcomes 



Research Partner3 



Feedback 



Production 
inputs; 

budget, staff, 

facilities, 
managerial 
Infrastructure 



Planning 
inputs: 

customer/ 
siakeholder jnpuls, 
surveillance and 
InLarvention 
effectiveness data, 
HHE's, earmarks, 
risk assessments 



Research': 

Surveillance, 
epidemiological anC 
behavioral slud-e«, 
rntervenlion studies, 
laborarory end field 
studies, 
eiposure 
measuremenla and 

risk assessment. 

control studies and 

devalapmeni, 

PPE iludies and 

development 



"iDiiamyral and 
aalramural, including 

domestic and 
intemalional efforts, 

such as worh 

conducted aiERCs, 

ARCS and WHO 

Global Network or 

Collaborating Centers 




-training and education 

materials and 

demonslrauon programs, 

irain^ profBcalpnals; 

- iDols and methods, best 
practices, developmental 
lechnologies, licenses, 
patervts 



Transfer: 

- translation ot 

research into 

practice, products' 

and technologies. 



Tton 



ation 



Ol lie mi nation, 

- capacity buiklir 

through techr 

assistance 

(HHE's), tram mo 

and education/ 



OSHA, MSHA, 

olher federal 

agencies, 

NIOSH 

prograrns; 

Congress; State 

& local 

agencies: 

SlancJards 

bodies: labor. 

trade and 

professional 

associalione: 

technology 

developers and 

menufacfurers; 

olher 

researchers; SH 

practitieners 





Pilot and/or 

market ready 

technologies, 

(raining and 

education 

programs, 

guidance, 

regulations, 

standards, 

trade and 

major media 



Employees. 

employers, 
industry, 

educators, 

regulators 

who reduce or 

prevent 

hazardous 
exposures or 

conditions 



improvements 
In safety and 

health In 
workplaces 
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FIGURE 1 The NIOSH operational plan presented as a logic model. 
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l.D Evaluation Committees' Information Needs 

Each NIOSH program under review will provide information to the relevant EC, 
including that outlined in Table 2. The EC may request additional information of 
NIOSH as needed, and NIOSH should provide it as quickly as is practical. 
NIOSH should consider organizing the information listed in Table 2 by subpro- 
gram or program as appropriate and to the extent possible. 

In addition to the information provided by NIOSH, the EC should independ- 
ently collect additional information that it deems necessary for evaluation {for ex- 
ample, the perspectives of stakeholders, such as OSHA, MSHA, unions and 
workforces, and industry). In conducting the review, the EC should continually 
examine how individual projects or activities contribute to the impact and rele- 
vance of a program as a whole. 

TABLE 2 Evaluation Committee Infomiation Needs 

• Program background and resources: 

o Program history. 

o Major program challenges. 

o Program strategic goals and objectives, past (for period under review) and cur- 
rent. 

o Major subprograms {if appropriate). 

o Results of previous program reviews (for example, annual review by NIOSH 
leadership team or external scientific program reviews). 

o External factors affecting the program. 

• Interactions with stakeholders and with other NIOSH programs: 

o The role of program research staff in NIOSH poliey-setting, OSHA and MSHA 
standard-setting, voluntary standard-setting and other government policy func- 
tions. 

o Interactions and working relationships with other NIOSH programs. 

o Identification of other institutions and research programs with overlapping or 
similar portfolios and an explanation of the relationship between NIOSH activi- 
ties and those of other instinjtions. 

o Key partnerships with employers, labor, other government organizations, aca- 
demic institutions, nonprofit organizations, and international organizations. 

• Program inputs: 

o Program resources (also called production inputs) 

■ Funding by year for period undei' review. 

■ Funding by objective or subprogram. 

■ Program staffing, FTE's, and laboratory facilities, by subprogram {if 
indicated). 

■ Percentage of program budget that is discretionary (beyond salaries). 

■ Percentage of program budget that is earmarked. 

■ Contributions from other agencies {in kind or funds), 
o Planning inputs 

■ Surveillance data, inputs from the Health Hazard Evaluation {HHE) or 
Fatality Assessment Control and Evaluation {FACE) program, or in- 
tramural and extramural research findings that influenced program 
goals and objectives. 

■ Planning inputs from stakeholders, for example, advisory groups, 
NORA teams, and professional, industry, and labor groups (specify if 
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any input from groups representing small business or vulnerable popu- 
lations). 

■ Related OSllA, Mine Safety and Health Administration (MSIiA) strategic 
plans, or other input. 

■ Proeess for soliciling and approving intramural research ideas. 

■ Process for soliciting and approving program-supported extramural re- 
search activities. 

• Program activities (more details provided in Table 3): 

D Intramural 

■ Surveillance activities. 

" Research activities (projects). 

■ Transfer activities to encourage implementation of research results for 
improved occupational safety and health (for example, information dis- 
semination, technical assistance, and technology and knowledge trans- 
fer). 

■ Key collaborations in intramural activities (for example, with other 
government agencies, academe, industry, and unions). 

o Extramural funded by NIOSH 

■ Requests for applications (RFAs) developed by program. 

■ Funded projects: grants, cooperative agreements, and contracts, such as 

Surveillance activities. 

Research activities. 

Transfer activities. 

Capacity-building activities. 

• Outputs (products of the research program — more details provided in Table 4): 

o Intramural 

■ Peer -reviewed publications, agency reports, alerts, and recommenda- 
tions. 

" Databases, Web sites, tools, and methods (including education and 
training materials). 

■ Technologies developed and patents. 

■ Sponsored conferences and workshops. 
o Extramural (to the extent practical), 

• Intermediate outcomes: 

o Standards or guidelines issued by other agencies or organizations based in whole 
or in part on NIOSI I research. 

o Adoption and use of control or personal protective technologies developed by 
NIOSH. 

o L^vidence of industry, employer, or worker behavioral changes in response to re- 
search outputs. 

o Use of NIOSH products by workers, industry, occupational health and safety 
professionals, health care providers, and so on (including internationally). 

o NIOSH Web-site hits and document requests. 

o Unique staff or laboratory capabilities that serve as a national resource. 

o Other intermediate outcomes. 

• End outcomes: 

o Data on program impact on rales and numbers of injuries and illnesses and ex- 
posures in the workplace (including trend data, if available). 
o Documentation of workplace risk reduction (quantitative, qualitative, ur both). 

• Dc!icription of current processes for setting research priorities and identifying 
emerging issues in the workplace. 
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I,E Prior Evaluations 

Several NIOSH programs have already been evaluated by internal and exter- 
nal bodies. The evaluations may have been part of an overall assessment of 
NIOSH, such as the 2005 Performance Assessment Rating Tool (PART) review/ 
or the evaluation of specific research program elements, such as any external sci- 
entific-program review. NIOSH should inform of, and the ECs should review, all 
prior evaluations of the program under review as an aid to understanding the evo- 
lution of the program and its elements. The EC evaluations, however, are inde- 
pendent of prior reviews and evaluations. 

II. SUMMARY OF EVALUATION PROCESS 

The ECs will assess the relevance and impact of NIOSH research programs. 
In conducting their evaluations, the ECs should ascertain whether NIOSH is doing 
the right things (relevance) and whether these things are improving health and 
safety in the workplace (impact). 

II.A The Evaluation Flowchart 
(Figure 2) 

To address its charge, the FC simplified the logic model of Figure 1 into a 
flowchart (Figure 2) that breaks the NIOSH logic model into discrete, sequential 
program components to be assessed by the EC. Each component of Figure 2 is 
addressed in greater detail in the indicated section of this document. The FC un- 
derstands that the activities of any research program will not be as linear as pre- 
sented in either Figures 1 or 2. The major components to be evaluated arc 

• Major program challenges. 

• Strategic goals and objectives. 

• Inputs (such as budget, staff, facilifies, the institute's research manage- 
ment, the NIOSH Board of Scientific Counselors, the NORA process, and 
NORA work groups). 

• Activities (efforts by NIOSH staff, contractors, and grantees, such as haz- 
ard surveillance; surveillance for injury, illness, and biomarkers of effect; 
exposure-measurement research; safety-systems research; injury- 
prevention research; health-effects research; intervention research; health- 
services research: and technology and knowledge transfer activities). 

• Outputs (NIOSH products, such as publications, reports, conferences, da- 
tabases, tools, methods, guidelines, recommendations, education and train- 
ing, and patents). 



The Part focuses on assessing program-level performance ;ind is one of Ihc measures of success of ihe 
budget and perfomiance inlegralion inilialive of the president's management agenda (see CDC Occupational 
Safely and Health al hup//www.whi!c'house.gov/oml>/h!iJ^ci/f}'2006/pma/hfis.pJ/). 
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• Intermediate outcomes (responses by NIOSH stakeholders to NIOSH 
products, such as public or private policy change, training and education 
in the form of workshop or seminar attendance, self-reported use or re- 
packaging of NIOSH data by stakeholders, adoption of NlOSH-developed 
technologies, implemented guidelines, licenses, and reduction in work- 
place hazardous exposure), 

• End outcomes (such as reduction in work-related injuries or illnesses or 
hazardous exposures in the workplace). 

The flowchart summarizes the PC's vision of how a program evaluation 
should occur. In evaluating each program or major subprogram, the EC must col- 
lect, analyze, and evaluate information on items described in each of the boxes of 
Figure 2, regardless of management structure (such as linear or matrix). The FC 
recognizes that the components of any program will not fit perfectly in any cate- 
gory in Figure I or 2. For example, training and development programs were ap- 
propriately defined as outputs by NIOSH in the logic model (Figure 1), but the FC 
finds more value in focusing on the responses to these outputs as intermediate 
outcomes (Figure 2, Box E) in the flowchart. The committee further recognizes 
that matrix organizations may have little control over the input portion of the 
logic model and that matrix program management may have fewer resources of its 
own on which to base its decisions. Following the suggested evaluation proce- 
dures, however, should ensure a desired level of consistency and comparability 
among all the ECs. 

Drawing on the program logic model, the flowchart, and EC members' exper- 
tise, the ECs will delineate important inputs and external factors affecting the 
NIOSH research program's agenda and the consequences of NIOSH research ac- 
tivities. Examples of external factors are research activities of industry and other 
federal agencies and the political and regulatory environment. For purposes of this 
review, the results of inputs and external factors are the program research activi- 
ties, outputs, and associated transfer activities that may result in intermediate out- 
comes and possibly end outcomes. 
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FIGURE 2 Flowchart for the evaluation of the NIOSH research program. 
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II. B Steps in Program Evaluation 

The FC concludes that useful evahmtion requires a disciphned focus on a small 
number of questions or hypotheses typically related to program goals, performance 
criteria, and performance standards; a rigorous method of answering the questions or 
testing the hypotheses; and a credible procedure for developing qualitative and quan- 
titative assessments. The evaluation process developed by the FC is summarized in 
Box 1 and described in detail in Section 111 of this document. 



BOX1 
The Evaluation Process 

1 . Gather appropriate information from NIOSH and other sources (see Table2). 

2. Determine timeframe to be covered in the evaluation {see III. B.I). 

3. Identify major program area challenges and objectives {see III.B,2). 

All NIOSH research programs are designed to be responsive to present or future 
workplace safety and health issues. Each research program should have its own ob- 
jectives. Each EC will provide an independent assessment of the major workplace 
health and safety problems related to the program under review and determine 
whether they are consistent with the program's stated goals and objectives. 

4. Identify subprograms and major projects in the research program. 

Each EC must determine how to disaggregate a program to achieve a manageable 
and meaningful evaluation of its components, and of the overall program. A program 
may need to be broken down into several recognizable subprograms or major pro- 
jects if an effective evaluation is to be organized. It may be advantageous for an EC 
to disaggregate a program into subprograms that NIOSH identifies. 

5. Evaluate the subprogram components sequentially (see III.B.2 through III.B.8), using 
the flowchart (Figure 2} as a guide. 

This will involve a qualitative assessment of each component of the research pro- 
gram. ECs will use professional judgment to answer questions and follow the guid- 
ance provided by the FC, 

6. Evaluate the research program's potential outcomes that are not yet appreciated (see 
I1I.B.9). 

7. Evaluate the important subprogram outcomes specifically for contributions to im- 
provements in workplace safety and health. 

Guidance is provided with specific items for consideration (see 111. B. 10). 

8. Evaluate and score the overall program for relevance (see III. B. 10). 

Final program ratings will consist of an integer score and discussion of its rationale. 

9. Evaluate and score the overall program for impact (see III. B. 10), 

Final program ratings will consist of an integer score and discussion of its rationale. 
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10, Identify success in targeting priority research and emerging issues (see lll.C). 
The EC should briefly discuss its assessment of the NIOSH program's process for 
determining priorities for research and emerging workplace issues. The ECs should 
also independently identify emerging workplace issues for which the NIOSH program 
under review should be prepared. 

1 1 . Prepare report by using the template provided in Section IV as a guide. 



III. EVALUATION OF A NIOSH RESEARCH PROGRAM— THE 

PROCESS 



HI. A Analysis of External Factors Relevant to the NIOSH Research Program 

As depicted in the logic model (Figure 1), reduction in injury and illness (end 
outcomes) or in exposure (intermediate outcome) is affected by stakeholder ac- 
tivities (external factors). Actions of those in labor, industry, regulatory entities, 
and others beyond NIOSH's control are necessary for the implementation of 
NIOSH recommendations. Implementation of research findings may depend on 
existing or future policy considerations. 

External factors may be considered as forces beyond the control of NIOSH 
that may affect the evolution of a program. External factors influence NIOSH's 
progress through all phases of the logic model and flowchart; from inputs to end 
outcomes (see Figures 1 and 2). Identification of external factors by an EC is es- 
sential because it provides the context for evaluation of the NIOSH program. Ex- 
ternal factors may be best assessed on the basis of the expert judgment of EC 
members who have knowledge of the field of research. Information regarding ex- 
ternal factors should also be sought from NIOSH, OSHA, and MSHA staff and 
from other stakeholders. The EC, however, may choose additional approaches to 
assess external factors. NIOSH should identity and describe external factors early 
in the evaluation sequence (see Table 2). Factors external to NIOSH might have 
been responsible for achieving some outcomes or might have presented formida- 
ble obstacles. The EC must address both possibilities. 

Some external factors may involve constraints on research activities related to 
target populations, methodologie issues, and resource availability. ECs might ex- 
amine whether 

• Projects addressing a critical health need are technologically feasible. 
However, a workforce of appropriate size and with appropriate duration 
and distribution of exposure for measuring a health effect may not exist; 
for example, no population of workers has been exposed for 30 years to 
fonnaldehyde at the eurrent OSHA permissible exposure level (PEL), so 
the related cancer mortality cannot yet be directly assessed. 

• Research is inhibited because NIOSH investigators are unable to access an 
adequate study population. Under current policy, NlOSll must either ob- 
tain an invitation by management to study a workplace or seek a judicial 
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order to provide authority to enter a worksite. {Cooperation under court 
order may well be insufficient for effective research.) 

• Research is inhibited because the work environment, materials, and his- 
torical records cannot be accessed even with management and workforce 
cooperation. 

• Adequate or established methods do not exist for assessing the environ- 
ment. 

• The NIOSH contribution to a particular field of research is reduced be- 
cause other institutions are working in the same field. 

• NIOSH resources are inadequate to tackle key questions. 

Evaluation of the impact of NIOSH research outputs on worker health and 
safety may require consideration of external factors that might impede or aide im- 
plementation, measurement, and so on. ECs might consider whether 

• Regulatory end points are unachievable because of obstacles to regulation 
or because of differing priorities of the regulatory agencies. For example, 
there may be no implementation of recommendations for improved respi- 
ratory protection programs for health-care workers because of enforce- 
ment policies or lack of acceptance by the health-care institution 
administrators. 

• A feasible control for a known risk factor or exposure is unimplemented 
because the costs of implementation are too high or because current eco- 
nomic incentives do not favor such actions. 

• End outcomes are unobservable because baseline and continuing surveil- 
lance data are not available. For example, the current incidence of occupa- 
tional noise-induced hearing loss is not known although surveillance for a 
substantial threshold shift is feasible. (NIOSH conducts surveillance of 
work-related illnesses, injuries, and hazards, but comprehensive surveil- 
lance is not possible with existing resources.) 

• Reductions in adverse effects of chronic exposure cannot be measured. 
For example, 90% of identified work-related mortality is from diseases, 
such as cancer, that arise only after decades of latency after first exposure; 
therefore, effects of reducing exposure to a carcinogen cannot be observed 
in the timeframe of most interventions. 

• A promulgated regulation requires a technology that was developed but 
not widely used. 

• Reductions in fatal traumatic injuries occur because more-hazardous 
manufacturing jobs are replaced by less-hazardous knowledge-based jobs. 



I II.B Evaluating NIOSH Research Programs by Using the Flowchart 

The FC used the NIOSH logic model {Figure 1) to define the scope and stages 
of an EC evaluation. The evaluation of the elements in the flowchart {Figure 2) 
summarizes the EC's vision of how a program evaluation should proceed. FC 
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members also identified numerous possible factors to eonsider in assessing the rele- 
vance of NIOSH research-program components, including 

• The severity or frequency of health and safety hazards addressed and the 
number of people at risk (magnitude) for these hazards. 

• The extent to which NIOSH research programs identify and address gen- 
der-related issues and issues of vulnerable populations. Vulnerable popu- 
lations are defined as groups of workers who have biologic, social, or 
economic characteristics that place them at increased risk for work-related 
conditions or on whom inadequate data have been collected. Vulnerable 
populations include disadvantaged minorities, disabled persons, low-wage 
workers, and non-Enghsh-speakers for whom language or other barriers 
present health or safety risks. 

• The extent to which NIOSH research programs address the health and 
safety needs of small businesses. 

• The "life stage" of problems being addressed. As the health effects are un- 
derstood, efforts should shift to intervention research, from efficacy to in- 
tervention, and to intervention-effectiveness research. Gaps in the 
spectrum of prevention need to be addressed; for example, research on ex- 
posure assessment may be necessary before the next intervention steps ean 
be taken. 

• The structure, in addition to the content, of the research program. A rele- 
vant research program is more than a set of unrelated research projects; it 
is an integrated program involving interrelated surveillance, research, and 
transfer activities. 

• Appropriate NIOSH considerafion of stakeholder input. 

The ECs may consider those and other important factors that bear on relevance as 
they progress through each stage of an evaluation. 

The following subsections are intended to guide the EC through the evaluation 
process and flowchart in Figure 2. Each begins with a definition of the component 
being evaluated, provides questions for the EC to consider during the course of its 
evaluation, and provides some guidance regarding the assessment of the compo- 
nent. The FC admittedly provides little guidance regarding the evaluation of pro- 
grams that arc organized in a matrix structure or programs that have large 
extramural research components. Because of the uniqueness of each program, 
each EC must detemiine the most reasonable way to apply the criteria established 
in this document. 



III. B.I Identifying the Period for Evaluation 

By studying materials presented by the NIOSH research program and other 
sources, the EC will become familiar with the history of the research program be- 
ing evaluated and its major subprograms, goals, objectives, resources, and other 
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pertinent information. Having that information, the EC should choose the period 
most appropriate for the evaluation. EC efforts should focus on the impact and 
relevance of the NIOSH program in the most recent appropriate period. As a start- 
ing point, the ECs might consider three general timeframes: 

• 1970-1995, the period from the founding of NIOSH to the initiation of 
NORA (pre-NORA period). 

• 1996-2005 (NORA 1 period). 

• After 2005 (NORA 2 period). 

Those timeframes are provided as general guidance; the period chosen for re- 
view will take into consideration suggestions from the NIOSH research program 
under review. It is recognized that many of the intermediate and end outcomes 
documented since 1996 are consequences of research outputs completed before 
1996. 



III.B. Identifying Major Challenges 
(Figure 2, Circle) 

Early in the assessment process, the EC itself should identify the major work- 
place health and safety challenges for the research program under review. In ar- 
riving at a list of challenges, the EC should rely on surveillance fmdings, 
including those of NIOSH investigations of sentinel events (through health-hazard 
or fatality-assessment programs), external advisory inputs, and its own expert 
judgment. The EC will then be able to compare its own assessment of workplace 
challenges with the NIOSH program goals and objectives. The congruence be- 
tween the two will be useful during the assessment of relevance. 



111.8.3 Analysis of Research-Program Strategic Goals and Objectives 

(Figure 2, Box A) 

The research program goals and objectives should be evaluated with a focus 
on how each program goal is related to NIOSH's agency wide strategic goals and 
to the program challenges identified in the step above (Section 1I1.B.2). The im- 
portance or relevance of an issue may differ from the influence of NlOSH-funded 
research in addressing it. The EC should recognize that NIOSH research priorities 
may be circumstantial (for example, congrcssionally funded) rather than based on 
NIOSH's assessment of the state of knowledge. 

Questions to Guide the Evaluation Committee 

1. Are the strategic goals and objectives of the program well defined and 
clearly described? 
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2. How well were program goals and objectives aligned with NORA 1 priori- 
ties during the last decade? 

3. How are current program strategic goals and objectives related to current 
NIOSH strategy, including NORA 2? 

4. Are the research program goals, objectives, and strategies relevant to the 
major challenges for the research program and likely to address emerging 
problems in the research program {as determined by the EC while address- 
ing Section [[[.B.2)? 

a. Did past program goals and objectives {research and dissemination and 
transfer activities) focus on the most relevant problems and anticipate 
the emerging problems in the research program? 

b. Do the current program goals and objectives target the most relevant 
problems? 

Assessment 

The EC should provide a qualitative assessment that discusses the relevance 
of the program's goals, objectives, and strategies in relation to its major chal- 
lenges. 



III. B.4 Review of Inputs 
(Figure 2, Box B) 

Planning inputs include input from stakeholders, surveillance and intervention 
data, and risk assessments. Production inputs include intramural and extramural 
funding, staffing, management structure, and physical facilities. 

The EC should examine existing intramural and extramural resources and, po- 
tentially, prior surveys or case studies that might have been developed specifically 
to assess progress in reducing workplace illnesses and injuries and to provide in- 
formation relevant to the targeting of research to future needs. The NIOSH re- 
search program should provide the EC all relevant planning and production inputs 
{sec below and Table 2 for examples). 

Planning inputs 

Planning inputs can be qualitative or quantitative. Sources of qualitative in- 
puts include 

• Federal advisory committees {such as the Board of Scientific Counselors, 
the Mine Safety and Health Research Advisory Committee, and the Na- 
tional Advisory Committee on Occupational Safety and Health). 

• NORA research partners, initial NORA stakeholder meetings, later NORA 
team efforts {especially strategic research plans), and the NORA Liaison 
Committee and federal liaison committee recommendations. 
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• Industry, labor, academe, professional associations, industry associations, 
and the Council of State and Territorial Epidemiologists (CSTE). 

• OSHA and MSHA strategic plans and other federal research agendas. 

Attention should be given to how comprehensive the inputs have been and to 
what extent gaps in input have been identified and considered by NIOSH. 
Sources of quantitative inputs include 

• Intramural surveillance information, such as descriptive data on exposures 
and outcomes (appropriate data may be available from a number of 
NIOSH divisions and laboratories). 

• HHEs. 

• Reports from the FACE program. 

• Extramural health-outcome and exposure-assessment data from OSHA, 
MSHA (both safety and health inspection data), the Bureau of Labor Sta- 
tistics, the US Department of Defense (DOD), and the US Department of 
Agriculture (USDA) (fatality, injury, and illness surveillance data); state 
government partners, including NIOSH-funded state surveillance pro- 
grams, such as Sentinel Event Notification System of Occupational Risks 
(SENSOR), Aduh Blood Lead Epidemiology and Surveillance (ABLES), 
and state-based FACE; and nongovernment organizations, such as the Na- 
tional Safety Council, the Association of Occupational and Environmental 
Clinics (AOEC), the American Society of Safety Engineers, and the 
American College of Occupational and Environmental Medicine. 

• Appropriate data from investigator-initiated extramural research funded by 
NIOSH. 

Production inputs 

For the research program under review, NIOSH should identify portions of the 
NIOSH intramural budget, staff, facilities, and management that play major roles 
in the research program. Production inputs should be described primarily in terms 
of intramural research projects, relevant extramural projects (particularly coopera- 
tive agreements and contracts), HHEs, and related staff. Consideration should also 
be given to leveraged funds provided by such partners as the National Institutes of 
Health (NIH) and the Environmental Protection Agency (EPA) for joint requests 
for applications or program announcements; and to OSHA, MSHA, and US De- 
partment of Defense (DOD) contracts with NIOSH. 

Assessment of inputs should include EC consideration of the degree to which 
allocation of funding and personnel was commensurate with the resources needed 
to conduct the research and the extent to which funding for Ihc relevant intramural 
research activity has been limited by lack of discretionary spending beyond sala- 
ries (travel, supplies, external laboratory services, and so on). Thus, assessments 
should consider the adequacy of the qualitative and quantitative planning and 
production inputs, given the tasks at hand. 
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Questions to Guide the Evaluation Committee 

1. Do planning, production, and other input data promote program goals? 

2. How well are major planning, production, and other program inputs used 
to support the major activities? 

3. Is input obtained from stakeholders, including input representing vulner- 
able working populations and small businesses? 

4. Are production inputs (intramural and extramural funding, staffing, man- 
agement, and physical infrastructure resources) consistent with program 
goals and objectives? 

Assessment 

The EC should provide a qualitative assessment that discusses the quality, 
adequacy, and use of inputs. 



IILB.5 Review of Activities 
(Figure 2, Box C) 

Activities are defined as the efforts and work of a program's staff, grantees, 
and contractors. For present purposes, activities of the NIOSH program under re- 
view arc divided into research and transfer activities. Table 3 is intended to guide 
the EC and NIOSH as to the type and organization of information required to 
evaluate program activities. The table may be incomplete, and some types of re- 
search activity may not be applicable to a given NIOSH program. Research activi- 
ties include safety research, health-outcomes research, safety-design research, and 
safety-systems research. Transfer activities include information dissemination, 
training, technical assistance, and education designed to translate research outputs 
into content and formats that are designed for application in the workplace. De- 
pending on the scope of the program under review, activities may also be grouped 
by research-program objectives or subprograms. 
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TABLE 3 Examples of NIOSH Program Research and Transfer Activities 

Surveillance (including hazard and injury, illness, and biomarkers or exposure or effect 
health surveillance and evaluation of surveillance systems) 

Health-effects research (illnesses, injuries, and biomarkers): 

Epidemiology 

Toxicology 

Physical and safety risk factors (laboratory -based) 

Development of clinical-screening methods and tools 

Exposure-assessment research: 

Chemical hazards 

Physical hazards 

Biologic hazards 

Ergonomic hazards 

Safety (traumatic injury) hazards 

Safer-design and safety-systems research 

lntcr\'Cntion research: 

Control technologies 

Engineering controls and alternatives 

Administrative controls 

Personal protective equipment 
Work organization 
Community participation 

Policy (such as alternative approaches to targeting inspections) 
Design for safety 
Emergency preparedness and disaster response 

Diffusion and dissemination research: 

Training effectiveness 
Information-dissemination effectiveness 
Difftision of technology 

llcalth-scrviccs and other research: 

Access to occupational health care 

Infrastructure — delivery of occupational-health services, including international health and 

safety 

Socioeconomic consequences of work-relaled injuries and illnesses 

Worker compensation 

Technology-transfer and other transfer activities: 

Information dissemination 
Training programs 
Technical assistance 



Conventional occupational safety and health research focuses appropriately on 
injury, illness, or death; on biomarkers of exposure; and on health effects of new 
technology, personal protective equipment, and regulations. A focus on surveil- 
lance research may be needed when available data inputs are inadequate. A focus 
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on socioeconomic and policy research and on diffusion research is also needed to 
effect, change because not all relevant intermediate outcomes occur in the work- 
place. NIOSH may be able to affect important outcomes farther out on the causal 
chain so as to influence health and safety in the workplace. Other research that 
might prove important in addressing NIOSH's mission includes 

• Surveillance research to assess the degree of significant or systematic un- 
derreporting of relevant injuries, illnesses, and biomarkers. 

• Socioeconomic research on cost-shifting between worker compensation 
and private insurance. 

• Research on methods to build health and safety capacity in community 
health centers that serve low-income or minority-group workers and to 
improve recognition and treatment of work-related conditions. 

• Transfer research to change health and safety knowledge of adolescents 
while they are in high school to improve the likelihood of reduced injuries 
as they enter the workforce. 

• Community- based participatory research on differences between recently ar- 
rived imrnigrants and US-bom workers regarding perceptions of acceptable 
health and safety risks so that programs can be targeted to meet the workforce 
training needs of immigrant workers. 

Transfer activities should be reviewed to determine whether the NIOSH pro- 
gram appropriately targets its outputs in a manner that will have the greatest im- 
pact. Ideally, information dissemination should be proactive, and strategic 
dissemination should be informed by research on the diffusion of new technolo- 
gies, processes, and practices. Highly relevant information and technology trans- 
fer should include plans for appropriate transfer to all appropriate worker 
populations, including those considered vulnerable. Training should be incorpo- 
rated into the strategic goals of all research fields where appropriate. 

The EC should review project-level research and transfer activities {including 
surveillance activities) that have been completed, are in progress, or planned by 
the program under review. The program under review should provide a list of ac- 
tivities and specify whether Ihey are intramural or extramural. For each extramu- 
ral project, the key organizations and principal investigators' names should be 
requested, as should whether the project was in response to a request for proposal 
or a request for application. For each intramural project, the EC should ask 
NIOSH to provide a list of key collaborators {from another government agency, 
academe, industry, or unions). 

The EC should evaluate each of the research activities outlined in Tabic 3 if it 
fomis an important element of the program research. In the case of a sector-based 
research program (for example, mining or constmction) in which health-effects 
research is not being reviewed, the EC should determine what research inputs in- 
fluence the program's strategic goals and objective, and then assess the value of 
the inputs. 
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Questions to Guide the Evaluation Committee in Assessing Research Activities 

1. What are the major subprograms or groupings of activities within the pro- 
gram? 

2. Are activities consistent with program goals and objectives? 

3. Are research activities relevant to the major challenges of the research 
program? 

a. Do they address the most serious outcomes? 

b. Do they address the most common outcomes? 

c. Do they address the needs of both sexes, vulnerable working popula- 
tions, and small businesses? 

4. Are research activities appropriately responsive to the input of stake- 
holders? 

5. To what extent are partners involved in the research activities? 

6. Are partners involved early in the research process so that they could par- 
ticipate in determining research objectives and research design? 

7. Were original resource allocations appropriate for the research activities, 
and do they remain appropriate? 

8. To what extent does peer reviews {internal, external, and mideourse) affect 
the activities? 

9. Is there adequate monitoring of quality-assuranee procedures to ensure 
credible research data, analyses, and conclusions? 

Questions to Guide the Evaluation Committee in Assessing Transfer Activities 

1 . Is there a coherent planned program of transfer activities? 

2. Are the program's information dissemination, training, education, techni- 
cal assistance, or publications successful in reaching the workplace or 
relevant stakeholders in other settings? How widespread is the response? 

3. To what degree have stakeholders responded to NIOSH information and 
training products? 

4. Is there evidence that the formats for information products were selected 
in response to stakeholder preferences? 

5. To what extent do program personnel rely on assessment of stakeholder 
needs and reactions to prototype information and training projects {forma- 
tive evaluation techniques)? 

6. To what extent docs the program build research and education capacity in- 
ternally and among stakeholders? 

Assessment 

For this part of the assessment, the EC will provide a qualitative assessment 
that discusses relevance. This assessment should include consideration of the ex- 
ternal factors identified in Section III.A that constrain choices of research projects 
and the relevance and effectiveness of transfer activities. The EC should consider 
the appropriateness of resource allocations. A highly relevant program would ad- 
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dress high-priority needs, produce high-quality results, be appropriately collabora- 
tive, be of value to stakeholders, and be substantially engaged in transfer activi- 
ties. A program may be less relevant to the extent that those key elements are not 
up to the mark or are missing. The discussion should cover those aspects in suffi- 
cient detail to arrive at a qualitative assessment of the activities. Assessment of 
the transfer activities must include considerations of program planning, coher- 
ence, and impact. The EC might also consider the incorporation of international 
research results into NIOSH knowledge- transfer activities for industry sectors in 
the United States. 



IILB.6 Review of Outputs 
(Figure 2, Box D) 

An output is a direct product of a NIOSH research program. Outputs may be 
designed for researchers, practitioners, intermediaries, and end-users, such as con- 
sumers. Outputs can be in the form of publications in peer-reviewed journals, rec- 
ommendations, reports, Web-site content, workshops and presentations, 
databases, educational materials, scales and methods, new technologies, patents, 
technical assistance, and so on. Outputs of the research program's extramurally 
funded activities should also be considered. Table 4 lists examples of major out- 
puts to be considered by the EC. The NIOSH research program should make 
every effort to include all pertinent data of the types listed in the table. 

Outputs may be tailored to the intended audience to communicate information 
most effectively and increase the likelihood of comprehension, knowledge, atti- 
tude formation, and behavioral intent. The extent of use of formative evaluation 
data {data gathered before communication for the purpose of improving the likcH- 
hood of the intended effects) and the extent of intended user feedback in the de- 
sign of the output can be considered indicators of appropriate quality assessment. 

TABLE 4 Examples of Research-Program Outputs to Be Considered 
Peer-reviewed publications by NIOSH staff; 

Number of original research articles by NIOSH staff 

Number of review articles by NIOSH staff (including best-praciiccs articles) 

Complete citation for each publication 

Complete copies of the ""top five" articles 

Collaboration with other public- or private-sector researchers 

Publications in the field of interest with other support by investigators also funded by NIOSII 
(for example, ergonomic studies with other support by an investigator funded by NIOSII 
to do ergonomics work, in which case NIOSII shoultl get some credit for seeding interest 
or drawing people into the field) 

Peer-reviewed publications by external researchers funded by NIOSH: 

Number of NIOSH-funded original research articles by external researchers 

Number of NlOSH-fimded review articles by external researchers (including best-practices 

articles) 

Complete citation for each written report 

Complete copies of the "top five" articles 

Collaboration with other government or academic researchers 
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NIOSH reports in the research program: 

Number ol" written reports 

Complete citalion for each written report 

Complete copies of the "top five" reports 

Sponsored conferences and workshops: 

Number of sponsored conferences 

Number of sponsored workshops 

Description of conferences and workshops (title, date, sponsors, target audience, number of par- 
ticipants, and 
resulting products) 

NlOSH's assessment of value or impact 

Databases: 

Number of major databases created by NIOSH staff 

Number of major databases created by external researchers funded by NIOSH grants 

Description of databases: 

Title, objective (in one to four sentences), and start and stop dates 
Partial vs. complete sponsorship (if partial, who were cosponsors?) 
Study or surveillance-system design, study population, and sample size 
Primary "products" of the database (such as number of peer-reviewed articles and re- 
parts) 

Complete copies of the "top two" publications or findings, to date, from each database 

Recommendations: 

Number of major recommendations 
Description of recommendations; 

Complete citation (article, report, or conference where recommendation was made) 

Summary in one to four sentences 

Percentage of target audience that has adopted recommendation I, 5, and 10 years later 

Up 10 three examples of implementation in the field 
Identification of "top five" recommendations to date 

Tools, methods, and technologies (TMT): 

Number of major TMT (includes training and educatioEi materials) 
Descriptions of TMT 

Title and objective of TMT (in one to four sentences) 

Complete citation (if applicable) 

Percentage of target audience that has used TMT 1,5. and 10 years later 

Up to three examples of implementation in the field 
Identification of "top 5" TMT to date 
Patents: 

Total number of patents 
For each: 

Title and objective (in one to four sentences) 

Complete citation 

Percentage of target audience that has used product 1,5, and 10 years later 

Up to three examples of implementation in the field 
Identification of "top five" patents to date 

Miscellaneous: 

Any other important program outputs 
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Some activities such as collaborations can also legitimately be conceptualized 
as outputs, because the collaboration itself is a result of NIOSH efforts. Coopera- 
tion, coordination, more intensive collaboration, and eventual formal partnering 
can be considered important outputs leading to desirable intermediate outcomes. 
Technology and knowledge transfer is greatly facilitated through such relation- 
ships. The extent of collaboration with other organizations in the determination of 
research agendas, the conduct of research, the dissemination of research results, 
and interorganization involvement in the production of outputs can all be meas- 
ures of output quality and quantity. The EC may consider coauthorship while try- 
ing to determine the importance of NIOSH research to the broader research 
community. 

The NIOSH program should provide information on all relevant outputs of the 
program under review produced during the chosen period. 

Questions to Guide the Evaluation Committee 

1 . What are the major outputs of the research program? 

2. Are output levels consistent with resources allocated (were resources allo- 
cated and used efficiently to produce outputs)? 

3. Does the research program produce outputs that address high-priority ar- 
eas? 

4. To what extent does the program generate important new knowledge or 
technology? 

5. Arc there widely cited peer-reviewed publications considered to report 
"breakthrough" results? 

6. What, if any, internal or external capacity-building outputs are docu- 
mented? 

7. Are outputs relevant to both sexes, vulnerable populations, and do they 
address health disparities? 

8. Are outputs relevant to health and safety problems of small businesses? 

9. Are products user-friendly with respect to readability, simplicity, and de- 
sign? 

10. To what extent docs the program help to build the internal or extramural 
insthutional knowledge base? 

1 1. Does the research produce effective cross-agency, cross-institute, or inter- 
nal-external collaborations? 

12. To what extent does the program build research and education capacity 
(internal or external)? 

Assessment 

The EC should provide a qualitative assessment discussing relevance and util- 
ity. The outputs of a highly ranked program will address needs in high-priority 
areas, contain new knowledge or technology that is effectively communicated, 
contribute to capacity-building inside and outside NIOSH, and be relevant to the 
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pertinent populations. The discussion should cover those aspects in sufficient de- 
tail to support the qualitative assessinent of the outputs. 



III.B. 7 Review of Intermediate Outcomes 
(Figure 2, Box E) 

Intermediate outcomes arc important indicators of stakeholder response to 
NIOSH outputs. They reflect the impact of program activities and may lead to the 
desired end outcome of improved workplace safety and health. Intermediate out- 
comes include the production by those outside of NIOSH of guidelines or regula- 
tions based wholly or partly on NIOSH research {products adopted as national or 
international public policy or as policy or guidelines by private organizations or 
industry); contributions to training and education programs sponsored by other 
organizations; use of publications or other materials by workers, industry, and oc- 
cupational safety and health professionals in the field; and citations of NIOSH 
research by industrial and academic scientists. 

Intermediate outcomes allow inference that a program's outputs are associated 
with observed changes in the workplace. Thus, an intermediate outcome reflects 
an assessment of worth by NIOSH stakeholders {such as managers in industrial 
firms) about NIOSH research or its products {for example, NIOSH training work- 
shops). Intermediate outcomes that arc difficult to monitor but may be valid indi- 
cators of relevance or utility include self-report measures by users of NIOSH 
outputs. Such indicators include the extent to which key intermediaries find value 
in NIOSH products or databases for the repackaging of health and safety informa- 
tion, the extent to which NIOSH recommendations are in place and attended to in 
workplaces, and employee or employer knowledge of and adherence to NIOSH- 
recommendcd practices. 

Questions to Guide the Evaluation ComiTiittee: 

1. Do program outputs result in or contribute to stakeholder training or edu- 
cation activities used in the workplace or in school or apprentice pro- 
grams? If so, how? 

2. Do program activities and outputs result in regulations, public policy, or 
voluntary standards or guidelines that arc transferred to or created by the 
workplace? 

3. Has the program resulted in changes in employer or worker practices as- 
sociated with the reduction of risk {for example in the adoption of new 
feasible control or personal protective technologies or administrative con- 
trol concepts)? 

4. Does the program contribute to changes in health-care practices to im- 
prove recognition and management of occupational health conditions? 

5. Docs the program result in research partnerships with stakeholders that 
lead to changes in the workplace? 
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6. To what extent do the program's stakeholders find value in NIOSH prod- 
ucts (as shown by document requests, Wcb-site hits, conference atten- 
dance, and so on)? 

7. Does the program or a subprogram provide unique staffer laboratory ca- 
pability that is a necessary national resource? If so, is it adequate, or does 
it need to be enhanced or reduced? 

8. Has the program resulted in interventions that protect both sexes, vulner- 
able workers, or address the needs of small businesses? 

9. To what extent did the program contribute to increased capacity at work- 
sites to identify or respond to safety and health threats? 

Assessment 

Only a qualitative assessment of product development, usefulness, and impact 
is required at this point in the EC report. Some thought should be given to the 
relative value of intermediate outcomes, and the FC recommends applying the 
well-accepted hicrarchy-of-controls model. The discussion could include com- 
ments on how widely products have been used or programs implemented. The 
qualitative discussion should be specific as to the various products developed by 
the program and the extent of their use by specific entities {industry, labor, gov- 
ernment, and so on) for specific purposes. Whether the products have resulted in 
changes in the workplace or in the reduction of risk should be discussed. The rec- 
ognition accorded to the program or the facilities by its peers {such as recognition 
as a "center of excellence" by national and international communities) should be 
considered in the assessment. To be highly ranked, a program should have high 
performance in most of the relevant questions in this section. An aspect of the 
evaluation can be whether the same changes in stakeholder activities and behav- 
iors would probably have occurred without NIOSH efforts. 



III.B.8 Review of End Outcomes 
(Figure 2, Box F) 

It is necessary for the EC to assess, to the greatest extent possible, NlOSH's 
contribution to end outcomes- — ^improvements in workplace health and safety 
(impact). For purposes of this evaluation, end outcomes are health-related changes 
that are a result of program acfivities, including decreases in injuries, illnesses, 
deaths and exposures or risk. Data on reductions in work-related injuries, ill- 
nesses, and hazardous exposures will be available for some programs, and in 
some eases they will be quantifiable. It is possible, however, to evaluate the im- 
pact of a NIOSH research program using either intermediate outcomes or end out- 
comes. If there is no direct evidence of improvements in health and safety, 
intermediate outcomes may be used as proxies for end outcomes in assessing im- 
pact as long as the EC qualifies its findings. The EC will describe the realized or 
potential benefits of the NIOSH program. Examples of realized intermediate out- 
comes are new regulations and widely accepted guidelines, work practices, and 
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procedures, all of which may contribute measurably to enhancing health and 
safety in the workplaee. 

The FC recognizes that assessing the causal relationship between NIOSH re- 
search and specific occupational health and safety outcomes is a major challenge 
because NIOSH does not have direct responsibility or authority for implementing 
its research findings in the workplace. Furthermore, the benefits of NIOSH re- 
search program outputs can be realized, potential, or limited to Ihc knowledge 
gained. Studies that conclude with negative results may nevertheless have incor- 
porated excellent science and contribute to the knowledge base. The generation of 
important knowledge is a recognized form of outcome in the absence of measur- 
able impacts. 

The impact of an outcome depends on the existence of a "receptor" for re- 
search results, such as a regulatory agency, a professional organization, an em- 
ployer, and an employee organization. The EC should consider questions related 
to the various stages that lead to outputs, such as these: 

1. Did NIOSH research identify a gap in protection or a means of reducing 
risk? 

2. Did NIOSH convey that information to potential users in a usable form? 

3. Were NIOSH research results (for example, recommendations, technolo- 
gies) applied? 

4. Did the applied results lead to desired outcomes? 

Quantitative data arc preferable to qualitative, but qualitative analysis may be 
necessary. Sources of quantitative data include 

• Bureau of Labor Statistics (BLS) data on fatal occupational injuries (the 
Census of Fatal Occupational Injuries) and nonfatal occupational injuries 
and illnesses (the annual Survey of Occupational Injury and Illnesses). 

• NIOSH intramural surveillance systems, such as the National Electronic 
Injury Surveillance System, the coal-worker x-ray surveillance program, 
and agricultural- worker surveys conducted by NIOSH in collaboration 
with USDA. 

• State-based surveillance systems, such as the NlOSlI-funded ABLES. and 
Ihc SENSOR programs (for asthma, pesticides, silicosis, noise-induced 
hearing loss, dermatitis, and burns). 

• Selected state worker-compensation programs. 

• Exposure data collected in the OSHA Integrated Management Information 
System. 

The FC is unaware of mechanisms for surveillance of many occupationally re- 
lated chronic illnesses, such as cancers that arise from long exposure to chemicals 
and other stressors. The incidence and prevalence of many such outcomes are best 
evaluated by investigator-initiated research. Research that leads to new, effective 
surveillance concepts or programs warrants special recognition. 
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The EC should recognize the strengths and weaknesses of outcome data 
sources. Quantitative accident, injury, illness, and employment data and databases 
are subject to error and bias and should be used by the EC only for drawing infer- 
ences after critical evaluation and examination of available corroborating data. 
For example, it is widely recognized that occupational illnesses are poorly docu- 
mented in the BLS Survey of Occupational Injuries and Illnesses, which captures 
only incident cases among active workers. It is difficult for health practitioners to 
diagnose work-relatedncss of most illnesses that may not be exclusively related to 
work; furthermore, few practitioners are adequately trained to make such an as- 
sessment. Many of those illnesses have long latencies and do not appear until 
years after people have left the employment in question. Surveillance programs 
may systematically undercount some categories of workers, such as contingent 
workers. 

In addition to measures of illness and injury, measures of exposure to chemi- 
cal and physical agents and to safety and ergonomic hazards can be useHil. Expo- 
sure or probability of exposure can serve as an appropriate proxy for disease or 
injury when a well-described occupational exposure-health association exists. In 
such instances, a decrease in exposure can be accepted as evidence that the end 
outcome of reduced illness or injury is being achieved. That is necessary particu- 
larly when the latent period between exposure and disease outcome, as in the case 
of asbestos exposure and lung cancer, makes effective evaluation of the relevant 
end outcome infeasible. 

As an example of how an exposure level can serve as a proxy, reduction in the 
number of sites that exceed an OSHA PEL or an American Conference of Gov- 
ernmental Industrial Hygienists threshold limit value is a quantitative measure of 
improvement of occupational health awareness and reduction of risk, in addition 
to exposure level, the number of people exposed and the distribution of exposure 
levels are important. Those data arc available from multiple databases and studies 
of exposure. Apart from air monitoring, such measures of exposure as biohazard 
controls, reduction in requirements for use of personal protective equipment, and 
reduction in ergonomic risks arc important. 

Challenges posed by inadequate or inaccurate measurement systems should 
not drive programs out of difficult fields of study, and the EC will need to be 
aware of such a possibility. In particular., contingent and informal working ar- 
rangements that place workers at greatest risk arc also those on which surveillance 
information is almost totally lacking, so novel methods for measuring impact may 
be required. 

The commitment of industry, labor, and government to health and safety are 
critical external factors. Several measures of that commitment can be useful for 
the EC: monetary commitments, attitude, staffing, and surveys of relative impor- 
tance. To the extent that resources allocated to safety and health are limiting fac- 
tors, the EC should explicitly assess NiOSH performance in the context of 
constraints. 
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Queslions lo Guide the Evaluation Committee 

1. What are the amounts and qualities of relevant end-outcomes data (such as 
injuries, illness, exposure, and productivity affected by health)? 

2. What are the temporal trends in those data? 

3. Is there objective evidence of improvement in occupational safety or 
health? 

4. To what degree is the NIOSH program or subprogram responsible for im- 
provement in occupational safety or health? 

5. If there is no time trend in the data, how do findings compare with data 
from other comparable US groups or the corresponding populations in 
other countries? 

6. What is the evidence that external factors have affected outcomes or out- 
come measures? 

7. Has the program been responsible for outcomes outside the United States 
that have not been described in another category? 

Assessment 

The EC should provide a qualitative assessment of the program and subpro- 
gram impact, discussing the evidence of reductions in injuries and illnesses or 
their appropriate proxies. 



lll.B.9 Review of Potential Outcomes 

There may be health and safety impacts not yet appreciated and other benefi- 
cial social, economic, and environmental outcomes as a result of NIOSH activi- 
ties. NIOSH study results may be influential outside the United States, and there 
may be evidence of implementation of NIOSH recommendations and training 
programs abroad. 

Questions to Guide the Evaluation Committee 

1. Is the program likely to produce a favorable change that has not yet oc- 
curred or not been appreciated? 

2. Has the program been responsible for social, economic, security, or envi- 
ronmental outcomes? 

3. Has the program's work had an impact on occupational health and safety 
in other countries? 

Assessment 

The EC may discuss other outcomes, including beneficial changes that have 
not yet occurred; social, economic, security, or environmental outcomes; and the 
impact that NIOSH has had on international occupational safety and health. 
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III.B.IO Summary Evaluation Ratings and Rationale 

The EC should use its expert judgment to rate the relevance and impact of the 
overall research program by first summarizing its assessments of the major sub- 
programs and then appropriately weighting the subprograms to determine the 
overall program ratings. 

Table 5 provides some background context to aid the EC in reaching overall 
ratings for relevance and impact. The EC could consider the items in Table 5 for 
each subprogram then for the overall program and assess the relevance of the re- 
search subprograms and program by reviewing earlier responses to the questions 
in Sections [1I.B.2 through II1.B.5 (reviews of program challenges, strategic goals 
and objectives, inputs, and activities). Items 1-4 in Tabic 5 are pertinent to assess- 
ing relevance. 



TABLE 5 Background Context for Program Relevance and Impact 
Assess the following for each subprogram: 

1 . Relevance of current and recently completed research and Iransfer activities to objective 
improvemenis in workplace safety and health. 

2. Contributions of NIOSH research and transfer activities to changes in work-related prac- 
tices and reduction in workplace exposures, illnesses, or injuries. 

3. Contributions of NIOSH research and transfer activities to improvements in work-related 
practices. 

4. Contributions of NIOSH research lo productivity, security, or environmental quality 
(beneficial side effects). 

5. Evidence of reduction of risk in the workplace (intermediate outcome). 

6. Evidence of reduction in workplace exposure, illnes.s. or injuries {end outcome). 

7. Evidence of external factors that prevented translation of NIOSH research results into in- 
termediate or end outcomes. 



To assess overall mipact, the EC first needs to consider the available evidence 
of changes in work-related risks and adverse effects and external factors related to 
the changes. The EC should review the responses to the questions in Sections 
111.B.6 through 111.B.8 (reviews of outputs, intermediate outcomes, and end out- 
comes) and systematically assess the impact of the research program and its sub- 
programs. Items 5-7 in Tabic 5 will be helpful. The EC should evaluate separately 
the impact of the research and the impact of transfer activities. Transfer activities 
occur in two contexts: NIOSH efforts to translate intellectual products into prac- 
tice and stakeholder efforts to integrate NIOSH results into the workplace. High 
impact assessments require the EC's judgment that the research program has con- 
tributed to outcomes; for example, outcomes have occurred earlier than they 
would have or arc better than they would have been in the absence of the research 
program, or outcomes would have occurred were it not for external factors be- 
yond NlOSli's control or ability to plan around. 
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The EC must assign one overall integer seorc for the relevance of the researeh 
program to the improvement of oeeupational safety and health and one overall inte- 
ger score for the impact of the program on the improvement of occupational safety 
and health. The EC will use its expert judgment, summary assessment of research- 
program elements, and any appropriate infomiation to arrive at those two scores. In 
light of substantial differences among the types of research programs that will be 
reviewed and the challenge to arrive at a summative evaluation of both relevance 
and impact, the PC chose not to construct an algorithm to produce the two final rat- 
ings. 

Relevance and impact scores will be based on fivc-poinl categorical scales es- 
tablished by the PC (see Boxes 2 and 3) in which 1 is the lowest and 5 the highest 
rating. The PC has made an effort to establish mutually exclusive rating catego- 
ries in the scales. When the basis of a rating fits more than one category, the high- 
est applicable score should be assigned. It is up to the EC to determine how 
individual subprograms should influence final scores. Single integer values should 
be assigned. Pinal program ratings will consist of integer scores for relevance and 
impact and prose jusfification of the scores. 

Box 2 includes the criteria for scoring the overall relevance of the NIOSH re- 
search program. As discussed in previous sections, numerous factors can be con- 
sidered in assessing relevance. The scoring criteria focus on two: the EC 
assessment of whether the program appropriately sets priorities among research 
needs and the EC assessment of how engaged the program is in appropriate trans- 
fer activities. Table 6 provides some guidance regarding how the EC may weight 
research priorities and transfer levels when determining relevance scores. 

The EC will consider both completed research and research that is in progress 
and related to likely ftiture improvements in its assessment of relevance. The EC 
should keep in mind how well the program has considered the frequency and severity 
of the problems being addressed; whether appropriate attention has been directed to 
both sexes, vulnerable populations, or hard-to-reach workplaces; and whether the dif- 
ferent needs of large and small businesses have been accounted for. It is up to the EC 
to determine how to consider external factors in assigning program scores. 
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BOX 2 
Scoring Criteria for Relevance 

5 = Research is in high-priority subject areas and NIOSH is significantly engaged in 

appropriate transfer activities for completed research projects/reported research re- 
sults. 

4 = Research is in priority subject areas and NIOSH is engaged In appropriate transfer 
activities for completed research projects/reported research results. 

3 = Research is in high priority or priority subject areas, but NIOSH is not engaged in 
appropriate transfer activities; or research focuses on lesser priorities but NIOSH is 
engaged in appropriate transfer activities. 

2 = Research program is focused on lesser priorities and NIOSH is not engaged in or 
planning some appropriate transfer activities. 

1 = Research program is not focused on priorities and NIOSH is not engaged in trans- 
fer activities. 



TABLE 6 Guidance for Weighting Research Priority and Engagement in Appropriate Transfer 
Activities in the Application of Relevance Score 

Engagement in Applicable 
Assessment of Research Priority Transfer Activities Applicable Score 



High priority 




Significantly engaged 


5 


High priority 




Engaged 


4 


High priority 




Not engaged 


3 


Priority 




Significantly engaged 


4 


Priority 




Engaged 


4 


Priority 




Not engaged 


3 


Lesser priority 




Significantly engaged 


3 


Lesser priority 




Engaged 


3 


Lesser priority 




Not engaged 


2 


Not focused on 


priorities 


Significantly 


2 


Not focused on 


priorities 


Engaged 


2 


Not focused on 


priorities 


Not engaged 


1 



Box 3 includes the criteria established for the rating of impact. In general, the 
EC will consider completed research outputs during the assessment of impact. In 
assigning a score for impact, it is important to recognize that a "major contribu- 
tion" (required for a score of 5) does not imply that the NIOSH program was 
solely responsible for observed improvements in worker health and safety. Many 
factors may be required to effect improvements. The EC could say that NIOSH 
made "major contributions" if the improvements would not have occurred when 
they did without NIOSH efforts. 
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The ¥C has some coneem that the imposed scoring criteria for impact might be 
considered a promotion of the conventional occupational-health research paradigm 
that focuses on health-etTects and technology research without much emphasis on the 
socioeconomic, policy, surveillance, and diffusion research (as opposed to diffusion 
activities) needed to effect change. The EC should remember that not all intermediate 
outcomes occur in the workplace. Important outcomes that NIOSH can effect also 
occur much farther out on the causal chain. NIOSH, for example, has an important 
role to play in generating knowledge that may contribute to changing norms in the 
insurance industry, in health-care practice, in public-health practice, and in the com- 
munity at large. The EC may find that some of those issues need to be addressed and 
considered as external factors that facilitate or limit application of more traditional 
research findings. Given the rapidly changing nature of work and the workforce and 
some of the intractable problems in manufacturing, mining, and some other fields, the 
EC is encouraged to think beyond the traditional paradigm. 



BOX 3 
Scoring Criteria for Impact 



5 = Research program has made major contribution(s) to worker health and safety on 
the basis of end outcomes or well-accepted intermediate outcomes. 

4 = Research program has made some contributions to end outcomes or well-accepted 
intermediate outcomes. 

3 = Research program activities are ongoing and outputs are produced that are lil<ely 
to result in improvements In worl^er health and safety (with explanation of why not 
rated higher). Well accepted outcomes have not been recorded. 

2 = Research program activities are ongoing and outputs are produced that may result 
in new knowledge or technology, but only limited application is expected. Well ac- 
cepted outcomes have not been recorded. 

1 = Research activities and outputs do not result in or are NOT likely to have any appli- 
cation. 

NA = Impact cannot be assessed; program not mature enou gh. 



IIJ.C Assessment of NIOSH Process for Targeting Priority Research Needs 
and Committee Assessment of Emerging Issues 

The second charge to the EC is the assessment of the research program's ef- 
fectiveness in targeting new research and identifying emerging issues in occupa- 
tional safety and health most relevant to future improvements in workplace 
protection. The EC is also asked to provide a qualitative narrative assessment of 
the program's efforts and to make suggestions about emerging issues that the pro- 
gram should be prepared to address. Among the most challenging aspects of re- 
search in illness and injury prevention are the identification of new or emerging 
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needs or trends and the formulation of a research response that appropriately uses 
scarce resources in anticipation of them. 

The EC should review the procedures that NIOSH and the research program 
have in place to identify needed research relevant to the NIOSH mission and 
should review the success that NIOSH has had in identifying and addressing re- 
search related to emerging issues. It should examine leading indicators from ap- 
propriate federal agencies, such as EPA, the Department of Labor, the National 
Institute of Standards and Technology, NIH, DOD, and the Department of Com- 
merce. Those indicators should track new technologies, new products, new proc- 
esses, and disease or injury trends. 

One source of information deserving particular attention is NIOSH HHE re- 
ports. The HHE program offers a potential mechanism for identifying emerging 
research needs that could be incorporated as input into each of the programs 
evaluated. The EC should determine whether the program under review appropri- 
ately considers pertinent HHE investigation findings. Additional emerging issues 
may be revealed through consideration of NIOSH and the NlOSH-funded FACE 
reports, the AOEC reports, the US Chemical Safety Board investigations, and 
SENSOR and other state-based surveillance programs. Appropriate federal advi- 
sory committees and other stakeholder groups should also be consulted to provide 
qualitative information. 

The EC should systematically assess how the research program and its subpro- 
grams target new research by evaluating each subprogram for the items listed in Ta- 
ble 7. The EC will have to determine how best to weight subprogram contributions in 
the program's targeting of new research. 



TABLE 7 Targeting of New Research and Identification of Emerging Issues 
Assess the following for each subprogram: 

1. Past and present effectiveness in targeting most relevant research needs. 

2. Effectiveness in targeting research in fields most relevant to fliture improvements in oc- 
cupational safety and health. 

3. Contribution of NIOSH research to enhancement of capacity in government or other re- 
search institutions. 



Questions to Guide the Evaluation Comwiltee 

1. What information docs NIOSH review to identify emerging research 
needs? 

a. What is the process for review? 

b. How often docs the process take place? 

c. How arc NIOSH staff scientists and NIOSH leadership engaged? 

d. What is the process for moving from ideas to formal planning and re- 
source allocation? 
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2. How arc stakeholders involved? 

a. What advisory or stakeholder groups are asked to identify emerging 
research targets? 

b. How often are such groups consulted, and how are suggestions fol- 
lowed up? 

3. What new research targets have been identified for future development in 
the program under evaluation? 

a. How were they identified? 

b. Were lessons that could help to identify other emerging issues learned? 

c. Does the EC agree with the issues identified and selected as important 
and with the NIOSH response, or were important issues overlooked? 

d. Is there evidence of unwise expenditure of resources on unimportant 
issues? 

The EC members should use their expert judgment both to evaluate the 
emerging research targets identified by NIOSH and to provide recommendations 
to NIOSH regarding additional research that NIOSH has not yet identified. Rec- 
ommendations should include a brief statement of their rationale. 



IV EVALUATION COMMITTEE REPORT TEMPLATE 

Consistency and comparability among EC report formats is desirable, but the 
FC recognizes that each NIOSH research program is different and that each EC is 
independent. The outline provided in Box 4 flows from the EC's review of 
NlOSH's generalized logic model {Figure 1), the evaluation flowchart {Figure 2), 
and the assessment model described earlier in this document. The EC should feel 
free to use or adapt this outline as necessary when organizing its final report. The 
FC encourages each EC to look at prior FC reports for organizational ideas. 
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BOX 4 
Suggested Outline for Evaluation Committee Reports 

I Introduction 

This section should be a brief descriptive summary of the history of the program (and 
subprograms) being evaluated with respect to pre-NORA, NORA 1 , and current and fu- 
ture plans of the research program presented by NIOSH. It should present the context for 
the research on safety and health; goals, objectives, and resources; groupings of sub- 
programs; and any other important pertinent information. (A list of the NIOSH materials 
reviewed should be provided in Appendix C.) 

II Evaluation of Programs and Subprograms (Charge 1) 

A. Evaluation summary {should include a brief summary of the evaluation with re- 
spect to impact and relevance, scores for impact and relevance, and summary 
statements). 

B. Strategic goals and objectives: should describe assessment of the program and 
subprograms for relevance. 

C. Review of inputs; should describe adequacy of Inputs to achieve goals. 

D. Review of activities: should describe assessment of the relevance of the activi- 
ties. 

E. Review of research-program outputs: should describe assessment of relevance 
and potential usefulness of the research program, 

F. Review of intermediate outcomes and causal impact: should describe assess- 
ment of the intermediate outcomes and the attribution to NIOSH; should include 
the likely impacts and recent outcomes in the assessment. 

G. Review of end outcomes: should describe the end outcomes related to health 
and safety and provides an assessment of the type and degree of attribution to 
NIOSH. 

H. Review of other outcomes: should discuss health and safety impacts that have 
not yet occurred; beneficial social, economic, and environmental outcomes; and 
international dimensions and outcomes. 

I. Summary of ratings and rationale. 

Ml NIOSH Targeting of New Research and Identification of Emerging Issues 
(Charge 2) 

The EC should assess the progress that the NIOSH program has made in targeting new 
research in occupational safety and health. The EC should assess whether the NIOSH 
program has identified important emerging issues that appear especially important in 
terms of relevance to the mission of NIOSH. The EC should respond to NIOSH's per- 
spective and add its own recommendations. 

IV Recommendations for Program Improvement 

On the basis of the review and evaluation of the program, the EC may provide recom- 
mendations for improving the relevance of the NIOSH research program to health and 
safely conditions in the workplace and the impact of the research program on health and 
safety in the workplace. 

Appendix A — Framework Document 

Appendix B — Methods and Information-Gathering 

Appendix C — List of NIOSH and Related Materials Collected in the Process of the 

Evaluation 
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Materials Provided by the 
NIOSH Health Hazard Evaluation Program 



NIOSH. 2007 (October 02). Evidence for the National Academies' review of the 
NIOSH Health Hazard Evaluation Program. Produced by the HHE Program 
staff for the Committee to Review the National Institute for Occupational 
Safety and Health (NIOSH) Health Hazard Evaluation (HHE) Program. Cin- 
cinnati, OH. 

NIOSH. 2007 (October II). HHE Program Stakeholders list that was provided as 
suggested stakeholders who could be ask to address the HHE Review Com- 
mittee. Cincinnati, OH. 

NIOSH. 2007 (November 14). NIOSH response #1 to questions for the NIOSH 
HHE Program from the Committee to Evaluate the NIOSH Health Hazard 
Evaluation Program. Cincinnati, OH. Included HHE Step Chart, HHE Organ- 
izational Chart, 3 Appendices, and 15 numbered and letter reports. 

NJOSH. 2007 (November 27). NIOSH response #2 to questions for the NIOSH 
HHE Program from the Committee to Evaluate the NIOSH Health Hazard 
Evaluation Program. Cincinnati, OH. 

NIOSH. 2007 (December 19 [revised]). NIOSH response #3 to questions for the 
NIOSH HHE Program from the Committee to Evaluate the NIOSH Health 
Hazard Evaluation Program. Cincinnati. OH. 

NIOSH. 2007 (December 21). NIOSH response #4 questions for the NIOSH HHE 
Program from the Committee to Evaluate the NIOSH Health Hazard Evalua- 
tion Program. Cincinnati, OH. 

NIOSH. 2008 (January 15). NIOSH response to #5 questions for the NIOSH HHE 
Program from the Committee to Evaluate the NIOSH Health Hazard Evalua- 
tion Program. Cincinnati, OH. Included NIOSH HHE decision process (triage 
flow chart). 

NIOSH. 2008 (January 28). NIOSH response to the request for more information 
about the outcomes of the flock HHEs from the Committee to Evaluate the 
NIOSH Health Hazard Evaluation Program. Included the flock section of the 
Respiratory Disease Research Program Evidence Package and journal article. 
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NIOSH. 2008 (February 5). NIOSH response to #6 questions for the NIOSH HHE 
Program from the Committee to Evaluate the NIOSH Health Hazard Evalua- 
tion Program. Cincinnati, OH. Included 8 HHE response letters. 

NIOSH. 2008 ( February 13). Compendiums from the HHE Program. Cincinnati. 
OH. 

NIOSH. 2008 (May 13). HHE Program's Procedures Manual. Cincinnati, OH. 
Included 42 different attachments. 

NIOSH. 2008 (June 3). NIOSH response to #7 questions for the NIOSH HHE 
Program from the Committee to Evaluate the NIOSH Health Hazard Evalua- 
tion Program. Cincinnati, OH. 
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This appendix provides additional detail regarding the methods used by the 
National Academies' Committee to Review the National Institute for Occupa- 
tional Safety and Health (NIOSH) Health Hazards Evaluation (HHE) Program to 
gather information to carry out its work. 



COMMITTEE MEETINGS 

As described in Chapter I, the committee held four meetings during the 
course of this study. The first three meetings included open sessions for informa- 
tion gathering. The agendas for these open sessions appear in Boxes C-I, C-2, and 
C-3 below. The fourth meeting was held in closed session. 



BOX C-1 

Meeting 1, October 18-19, 2007 

The National Academy of Sciences Building 

2101 Constitution Avenue 

Wastiington, DC. 20418 

OPEN SESSION AGENDA 
Thursday, October 18 

NIOSH BRIEFINGS: The HHE Program 

10:45 a.m. Study Context and Goals, NIOSH Perspective {Lewis Wade, Pti.D., Sen- 

ior Science Advisor) 

11:00 a.m. HHE Overview: Introduction, Strategic Goals, and Evaluation {Teresa 

Scfinorr, Pfi.D., Director, Division of Surveillance, Hazard Evaluations, 
and Field Studies) 

11:20 a.m. HHE Overview: Management, Activities, and Outcomes {Allison Tepper, 

Ph.D., Chief, Hazard Evaluations and Technical Assistance Branch 

[HETAB]) 
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11:45 a.m. 



Lunch 



NIOSH BRIEFINGS: HHE PROGRAM (continued) 

12:45 p.m. HHE Program Goal 1: Prevent occupational illnesses througti reduced 

exposure to workplace hazards 

• Biological Hazards {Teresa Seitz, M.P.H, C.I.H, Supervisory Industrial Hygiene 
Team Lead, HETAB) 

• Chemical Hazards {Teresa Seitz) 

• Physical Hazards {Ken Wallingford ,CJ.H, Deputy Chief, HETAB) 

• Mixed Hazards {Ken Wallingford) 

1.25 p.m. Question-and-Answer Session: Committee and NIOSH Briefers 

1 :40 p.m. HHE Program Goal 2: Promote occupational safety and health research 

on emerging issues {Kay Kre/ss, M.D., Chief, Field Studies Branch) 

2:00 p.m. Question-and-Answer Session: Committee and NIOSH Briefers 

2:15 p.m. Break 

2:25 p.m. HHE Program Goal 3: Protect the health and safety of workers during 

public health emergencies {Bruce Bernard, M.D., M.P.H. , Supervisory 
Medical Teann Lead, HETAB) 

2.45 p.m. Question-and-Answer Session: Committee and NIOSH Briefers 

3:00 p.m. HHE Program Future Directions {Allison Tepper) 

3:10 p.m. Closing Question-and-Answer Session: Committee and NIOSH Briefers 

3:30 p.m. PUBLIC COMMENT SESSION 

4:00 p.m. End of Open Session 



BOX C-2 

Meeting 2, December 10-11, 2007 

The Keck Center of the National Academies 

500 Fifth Street, NW 

Washington, D.C. 20001 

OPEN SESSION AGENDA 
Monday, December 10 

10:15 a.m. Public Welcome and Brief Committee Member Introductions {Rogene 

Henderson, Chair) 

10:20 a.m. Committee's task and importance of stakeholder input 
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STAKEHOLDER PANELS 

Each session will include a question and answer session among committee and 

discussants 

10:30 a.m. Workplace Sector Panel 

Moderator : Rosemary Sokas, committee member 

Discussants : 

Shelley Davis, Farmworker Justice, Washington, DC 

Janle Gittleman, Center to Protect Workers' Rights, Silver Spring, MD 

Jose Olive, Inlerfaith Worker Justice, Chicago, IL 

Frank Renshaw, Rohm and Haas Company, Corydon, PA 

11:45 a.m. Lunch 

1 :00 p.m. Public Sector Panel 

Moderator : Barbara Silverstein, committee member 
Discussants : 

Marthe Kent, OSHA Region 1, Boston, Massachusetts 

Andrea Kidd-Taylor, Community Health and Policy, Morgan State University, Balti- 
more 

Kenneth Rosenman, Occupational and Environmental Medicine, Michigan State Uni- 
versity, East Lansing 
Joshua Sharfstein, Baltimore City Health Department 

2:15 p.m. Break 

DISCUSSION WITH NIOSH 

2:30 p.m. Discussion and clarification of NIOSH responses to committee questions 

posed after the first meeting 

4:00 p.m. End of open session 



BOX C-3 

Meeting 3, January 15-16, 200B 

Arnold and Mabel Beckman Center of the National Academies, Board Room 
100 Academy Drive Irvine, CA 92617 

OPEN SESSION AGENDA 
Tuesday, January 15 

10:15 a.m. Public Welcome and Brief Committee Member Introductions (Rogene 

Henderson, Committee Chafr) 

1 0:20 a.m. Committee's task and importance of stakeholder input 

10:30 a.m. Input regarding HHE conducted for the Alameda county Public Authority 

for In-Home Services (HETA # 2001-0139-2930) 
Linda Ayela, Training and Outreach Coordinator, Public Authority for In-Home sup- 
portive Services in Alameda County, CA (via telecom) 



Prepublication Version ~ Subject to Further Editorial Revision 



^(^ 



Till- HI^.-ILTII IIAZ.'IRD EVALUATION PROG/t-iM AT mOSH 



1 1 :00 a.m. Input regarding HHEs conducted at Kaiser Permanente (HETA-2003- 

0280-2974) 
Peggy Hoffman, Kaiser Permanente, Redwood City, CA 
Barbara Smisko. Kaiser Permanente, Santa Teresa, CA (via telecom) 

11:30 a.m. Input regarding HHE conducted at U.S. Magnesium (HETA #2004-0169- 

2982) 
Tom Tripp, Technical Services Coordinator, U.S. Magnesium, Rowley, Utah {via tele- 
com) 
Bryant Hardy. Union Representative, U.S. Magnesium, Rowley, Utah 
Mike Wriglit, Director, Health Safety and Environment, United Steelworkers of Amer- 
ica 
12:00 Lunch 

1:00 p.m. Barbara Materna, PhD, Chief, Occupational Health Branch, California 

Department of Public Health 

1 :30 p.m. Input regarding HHE conducted at Carmi Flavor and Fragrance Com- 

pany (HETA-2006-0303-3043) 
Roger Speakman. VP Manufacturing, Carmi Flavor and Fragrance Company 
Barbara Materna, PhD, Chief, Occupational Health Branch, California Department of 
Public Health 

2:00 p.m. Opportunity for Comments from NIOSH 

2:30 p.m. End open session 



COMMITTEE REQUESTS FOR INPUT 

To receive a broader range of stakeholder input, the evaluation committee re- 
quested pubhc input via an online questionnaire (see Chapter 1 for more detailed 
discussion). The questionnaire was announced via email to over 500 stakeholders, 
and responses eould be submitted online, by e-mail, or standard mail. The option 
to respond anonymously was available. A second questionnaire specifically re- 
lated to HHE Program emergency response activities was also distributed. The 
questionnaires are included as Boxes C-4 and C-5. A summary of responses ean 
be found in Appendix D. 



BOX C-4 

Text of Broadcast Questionnaire 

Opportunity for fnput to the National Academies Committee to Review the NIOSH 
Health Hazard Evaluation Program 

Dear Colleague: 

The Committee to Review the National Institute for Occupational Safety and Health 
(NIOSH) Health Hazard Evaluation (HHE) Program was formed by the National Research 
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Council to review the impact, relevance, and future directions of the program. The study 
is being conducted under the auspices of the Division of Earth and Life Studies. The 
committee's first meeting was held October 18-19, 2007. Your input would be valuable for 
consideration at future meetings on December 10-11, 2007, and January 15-16, 2008. 

In addition to evaluating what the HHE Program produces, the committee will determine 
whether it is appropriate to credit the program with changes in worl<place practices, haz- 
ardous exposures, and/or occupational illnesses and injuries, or whether the changes are 
the result of other unrelated factors. In conducting its assessment, the committee will 
evaluate 

A. Theimpact of the HHE Program on 

• Reducing worl<er risk and preventing occupational Illness in investigated workplaces; 

• Transferring program-generated Information to relevant employers and employees 
beyond the investigated workplaces; 

• NIOSH research and policy-development programs; and 

• The activities of regulatory agencies, occupational safely and health professionals 
and organizations, state and local health agencies, and others in the occupational 
health community, as achieved by transferring program generated hazard and pre- 
vention information 

B. The relevance of the HHE Program in addressing current and emerging workplace 
health hazards. 

The committee has generated a list of questions (see below) to guide your input. 

QUESTIONS 

1. Are you familiar with the NIOSH Health Hazard Evaluation (HHE) Program activities 
and reports related to occupational safety and health? 

2. How did you find out about the HHE Program (e.g., website, a recommendation)? 

3. If you have made an HHE request, what kind of experience have you had? Please be 
explicit in your response. 

4. Have you ever considered requesting an HHE and ultimately decided against making 
the request? If so, why? 

5. Do you have any suggestions about improving access to Informalion about the pro- 
gram? 

6. What are the most serious occupational health hazards you have encountered in 
your work, and how could the HHE Program help you address these issues? 

7. What do you see as significant emerging health hazards in occupational safety and 
health? 

8. What is your occupation or to what type of organization or industry to you belong? 
Please feel free to provide your name (optional). Any information you provide will 
help the committee understand the context for your responses. 



Prepublication Version - Subject to Further Editorial Revision 



178 



THE HEALTH HAZARD EVALUATION PROGRAM AT NJOSH 



BOX C-5 



Text of Emergency Response Questionnaire 



Dear Colleague: 



The Committee to Review the National Institute for Occupational Safety and Health 
(NIOSH) Health Hazard Evaluation (HHE) Program was formed by the National Research 
Council to review the impact, relevance, and future directions of the program. The study 
is being conducted under the auspices of the Division of Earth and Life Studies. The 
committee has held meetings October 18-19, 2007, December 10-11, 2007, and January 
15-16, 2008. The committee's final meeting will be held February 21-22, 2008. The com- 
mittee requests your input as a stakeholder of the HHE Program's emergency response 
activities. 

In addition to evaluating what the HHE Program produces, the committee will determine 
whether it is appropriate to credit the program with changes in workplace practices, haz- 
ardous exposures, and/or occupational illnesses and injuries, or whether the changes are 
the result of other unrelated factors. In conducting its assessment, the committee will 
evaluate 

A. The impact of the HHE Program on 

• reducing worker risk and preventing occupational illness in investigated workplaces; 

• transferring program-generated information to relevant employers and employees 
beyond the Investigated workplaces; 

• NIOSH research and policy-development programs; and 

• the activities of regulatory agencies, occupational safety and health professionals 
and organizations, state and local health agencies, and others in the occupational 
health community, as achieved by transferring program-generated hazard and pre- 
vention information. 



B. The relevance of the HHE Program in addressing current and emerging workplace 
health hazards. 



You can learn more about the committee and its charge at the following website: 
http://wviAv8.nationalacademies.org/cp/projectview.aspx?key=48818 

The committee has generated a list of questions (see below) to guide your input. If you 
wish to contribute, please respond via email to me at your earliest convenience. Keep in 
mind that any written comments submitted to the committee are required to be included 
in the study's public access file. If you wish your comments to remain anonymous, the 
committee has developed an on-line questionnaire in which providing your name and 
affiliation is optional (http://viAww8.nationalacademies.org/survey/dels/nioshhhe.htm). The 
questions on the website, however, were not specifically designed to learn about HHE 
Program emergency response activities as the questions below are. 

Feel free to respond as you see most appropriate and to forward this request for input to 
other professionals. 
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Questions regarding NIOSH HHE Program emergency response activities 

1. What was the nature of your experience with NIOSH HHE emergency response staff 
who worked with your agency? 

2. What was the NIOSH HHE staff role in the investigation or response? Was it rele- 
vant and/or appropriate in your opinion? How could it have been changed? What 
would you suggest doing in the future if similar circumstance arose? 

3. What do you feel was the overall quality of work performed by the HHE staff re- 
sponders? Was their arnval timely? Were reports prepared in a timely fashion? 

4. What was the overall impact of the NIOSH HHE staff involvement in the investigation 
or emergency response? Please address specifically the HHE Program's impact 
on protection of workers from health hazards during the incident/investigation. To 
what extent were they successful? Were there any measurable changes in proc- 
ess/procedures/worker protection as a result of HHE Program staff involvement? 

5. What is your occupation or to what type of organization or industry do you belong? 
Please feel free to provide your name (optional). Any information you provide will 
help the committee understand the context for your responses. 



Prepublication Version - Subject to Further Editorial Revision 



Appendix D 

Stakeholder Input: 
Key Recommendations and Emerging Health Hazards 



RECOMMENDATIONS FROM STAKEHOLDERS REGARDING THE 

HHE PROGRAM 

In response to requests for input via an online questionnaire (see Chapter I 
and Appendix C) and during discussions at committee meetings, stakeholders 
made useful comments and suggestions about the Health Hazard Evaluation 
(HHE) Program. The comments are summarized in this appendix. A detailed table 
recording the text of all original responses is available through the Public Access 
Records Office of The National Academics (Stakeholder Response Table, 2008). 

The following list represents input from occupational, environmental, and in- 
ternal physicians, medical directors, and nurses; environmental safety specialists 
and administrators; industrial hygienists; toxicologists; epidemiologists; research 
psychologists; engineers; academics; compliance officers; investigators; nonprofit 
health organization representatives; union and worker representatives; and federal 
and state representatives. Some of the respondents work or have worked with the 
National Institute for Occupational Safety and Health (NIOSH) or the Occupa- 
tional Safety and Health Administration (OSHA). Some have had direct experi- 
ence with the HHE Program, either through work experience or through requests 
made for investigations at worksites. 

The comments and recommendations have been divided into several catego- 
ries of suggested HHE Program improvements: general, conducting evaluations, 
evaluation reports, product dissemination and outreach, and training. Comments 
related to emerging issues arc summarized in the final section of this appendix. It 
should be noted that the comments do not reflect the views or opinions of either 
individual committee members or the committee as a whole, although they may 
have influenced deliberations which led to the committee's conclusions 
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HHE Program Improvements 
General 

• Improve the connection between NiOSH management and the front-line sci- 
ence staff to allow the program to become more science oriented. 

• Encourage state-based NOISH programs to make referrals for HHEs. 

• Assist OSHA in being more specific in rulemaking by making recommenda- 
tions for controls cither listed in the order of effectiveness or combinations, 
and by supplying priority listing of the literature. 

• Meet specific needs of all types of workplaces, but prioritize the hazards stud- 
ied. 

• Target investigations to certain industries, such as poultry and restaurant. 

• Have rapid response teams to encourage timeliness in response to HHE re- 
quests. 

• Have more direct telephone contact with the workplace or requestor to en- 
courage timely HHE reports, as well as better explanations of the criteria for 
onsite visit responses. 

• Because the range of federal, state, and local agencies that could respond is 
confusing, engage local experts in nearby field offices as consultants. 

• Create a national surveillance system for occupational illness. 

• Look to workers compensation for trend data. 

• Consult with OSHA for small- and mid-sized organizations in particular, and 
encourage employers with an evaluation program to use the consultation pro- 
gram. 

• Strengthen ties between NIOSH and state health departments (e.g., funding, 
cooperative agreements) to create surveillance programs in as many states as 
possible. 

• Collaborate nationally with federally designated pediatric environmental 
health specialty units (for example, Centers for Disease Control and Preven- 
tion [CDC], U.S. Environmental Protection Agency) to conduct HHEs in day 
care and schools. 

• Work with the Chemical Safety Board to address these hazards 

• Actively solicit reports of unusual illnesses or patterns of illnesses among 
workers from occupational physicians, companies, and workers. 

• Collect and study the trend data to determine sentinel events. 



Conducting Evaluations 

• Make suggestions at the time of worksite investigations, and provide interim 
reports as soon as possible. 

• Encourage the use of pollution prevention or toxic use reduction strategies to 
reduce hazards at the source. 
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• Improve adherence to notification protocol for local public health agencies 
when an HHE is scheduled in a given state to promote better access to rele- 
vant historic and recent testing data. 

• Have more sophisticated neuropsychological testing of workers exposed to 
neurotoxic metals, solvent, and pesticides. 

• To be able to determine reduction in worker risk as a result of HHEs, follow 
up on an ongoing basis with individual workplaces and requestors to track the 
feasibility and implementation of recommendations. 



Evaluation Reports 

• Prepare better short summaries of completed work products. 

• Prepare both short and full reports. 

• Prepare one-page summaries in layperson language for more wide distri- 
bution of the reports. 

• Add a paragraph in each report that it is the expectation that each injury 
and its causes would be investigated, and that employers would identify 
and implement solutions to prevent recurrence. 



Product Dissemination and Outreacli 

• Make search engine improvements, with alphabetical categorization by 
main topic or states, and grouping by type of contaminant or hazard for 
cross referencing, in addition to search by terms. 

• Link HHE reports to OSHA and NIOSH webpages more; include an 
online, user-friendly search mechanism at the CDC website. 

• Post HHE Program success stories on the OSHA website. 

• Collaborate with relevant professional associations and employer or busi- 
ness groups (e.g.. National Business Group on Health). 

• Partner with community institutions, worker advocate groups, faith-based 
centers, and day labor organizations to relate to immigrant workers. 

• Distinguish NIOSH from the Immigration and Customs Enforcement and 
other government agencies in publication and other public information. 

• Post online factshcets and other materials for workers in languages other 
than English. 

• Reach out to state and local public-health officers to promote better pub- 
lic-health measures. 

• Provide online presentations to elicit commentaries from safely and health 
practitioners. 

• Promote the HHE Program to unions, workers (organized and unorgan- 
ized), businesses, nongovernmental organizations, occupational and envi- 
ronmental health professionals, healthcare providers, and state and local 
health departments. 



Prepiihlication Version - Subject to Further Editorial Revision 



184 THE UEALTiUIAZARI) EVALUATION PROGRAM AT NIOSH 

• Target the eonstruction seetor beeausc of its limited resources and high 
risks in both safety and health issues. 

• Ensure that OSHA complianee officers are informed about the HHE Pro- 
gram and how it ean be aeeessed by employees. 

• Post information in all workplaecs. 

• Create pamphlets for offices of health and safety to provide information 
for all stakeholders about HHEs. 

• Advertise in local newspapers. 

• Publicize more, including dissemination through trade organizalions and 
their journals throughout the business sector, and trade and union fairs and 
conferences. 

• Bring more attention to publication of sentinel new findings, case reports, 
analyses of trends, and so forth, in peer-reviewed professional journals, 
including the publication of summaries of HHE reports of particular inter- 
est. 

• Educate healthcare facilities about exposures in the hospital environment 
and collaborate with the NlOSH research arm lo follow up on issues. 

• Reference timely "one-liners" in Morbidity and Mortality Weekly Reports 
and include specific web addresses to link to the HHE report 

• Send reports to state and local government agencies, and to for- and non- 
profit organizations. 

• Advertise through better business bureaus, wholesale marketers, and day 
labor organizations and Telemundo to reach immigrant populations. 

• Communicate better with safety professionals in industry. 

• Provide subscriptions to selective e-mail HHE-specific brief reports (e.g., 
NlOSH cNews). 

• Market to faculty and students of NIOSH-funded Education and Research 
Centers. 



Training 



Provide field placement training in NlOSH for students and inform profes- 
sors about the HHE Program to help promote the field of occupational 
health. 

Make it an OSHA requirement to have training related to the HHE Pro- 
gram. 



Emerging Health Hazards 

Stakeholders pointed to a number of emerging health hazards. Broad com- 
ments about emerging health hazards included concern about all-hazards disaster 
preparedness, emergency response, and terrorist-related defense or assessments. 
Also mentioned were unforeseen issues associated with climate change, water 
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shortages, and other environmental extremes. Continuing development of new 
compounds and chemicals, including the use of chemicals about which there are 
limited or no health data, was brought up as a conecm. 

The downsizing of health and safety staffs across the country and weak en- 
forcement of safety and health laws, as well as emerging issues related to the ag- 
ing workforce were items of concern. The identification of broad causes of injury 
in each plant and industry was indicated as an emerging issue, as well as devel- 
opment of targeted prevention and intervention programs to create a national ex- 
pectation that each injury and its causes would be investigated, and that employ- 
ers would identify and implement solutions to prevent recurrence. 

More focused workplace examples of emerging health hazards included the 
effect of supervisor abuse and work stress in the reporting of problems in the 
workplace, and organization of work, workload, and stress, including shifts, shift 
work, and its consequences; cultural issues, expectations, and productivity; com- 
munication; civility, including appropriate public behaviors, were noted; and 
workplace violence. Further, unrecognized or uncharacterized respiratory hazards 
related to work organization, job stress, psychosocial disorders, and muscu- 
loskeletal disorders were mentioned. 

Other wide-ranging, specific emerging hazards included impulse noise; occu- 
pational exposure limits; serum chromium/beryllium; lead; silica; solvents; radia- 
tion; dermatitis; fiberglass exposure; nanotcchnology/nanoparticle and hexa 
chrome exposures; surgical smoke plume from all types of sources; chemo and 
hazardous drug exposures; indoor environmental quality issues, including water 
damaged buildings, mold growth, and poor air quality; synthetic biology creation 
of biofuels; use of ingredients in popcorn manufacturing facilities; ergonomic is- 
sues related to text messaging and small computer use; respiratory disease and 
exposures in many industries; neurotoxicants. including their effect on pregnant 
women and those who are breastfeeding; unregulated or underegulatcd toxics 
identified as carcinogens; sugar dust; welding fumes; coal dust; musculoskeletal 
issues; ergonomic issues with multiple operators of the same piece of equipment; 
bioaerosol exposure and organic dust; effects of food additives; hazards associ- 
ated with construction and warehousing; pharmaecutieals/bioteehnology; agricul- 
tural hazards; emerging infectious diseases; sources of autoimmune diseases in 
women, and healthcare provider immune system problems; multiple ehemieal ex- 
posure during and after hardwood floor finishing; toxic molds; mycotoxins; en- 
dotoxins: bacteria (e.g., methicillin-rcsistant Staphylococcus aureus); pesticide 
application; fonnaldehyde in particle board furniture; needlestieks; mixed low- 
level exposures; cleaning products (e.g., disinfectants); hexavalent chromium in 
Portland cement and concrete; diescl exhaust; work-related asthma; gasoline gen- 
erators and gas-powered washers; and golf ball manufacture and rubber coaling. 
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Committee Members 

ROGENE F. HENDERSON, Ph.D., (Chair) is a senior biochemist and toxi- 
cologist emeritus in the Experimental Toxicology Program of the Lovelace Respi- 
ratory Research Institute, and a clinical professor in the College of Pharmacy at 
the University of New Mexico in Albuquerque. Her major research interests arc in 
the use of bronchoalveolar lavage fluid analyses to detect and characterize bio- 
markers of developing lung disease, the toxicokinetics of inhaled vapors and 
gases, and the use of biological markers of exposure and of effects to link envi- 
ronmental exposure to disease. She has served on a number of DOE, EPA, 
NIEHS, and U.S. Army scientific advisory boards, and was recently appointed 
chair of EPA's Clean Air Scientific Advisory Committee. Dr. Henderson is a na- 
tional associate of The National Academies who has chaired and served as a 
member of a number of committees, including as a member of the Board on Envi- 
ronmental Studies and Toxicology. She received her Ph.D. in chemistry from the 
University of Texas. 

JOEL BENDER, M.D., Ph.D., has been corporate medical director for General 
Motors (GM) Health Services since May 2002 where he directs GM's global oc- 
cupational health programs and policies. He also acts as a health-care consultant 
for GM, directs work life programs for salaried employees, serves as a govern- 
ment liaison with health-related agencies, and is responsible for health promotion 
activities for GM employees and their family members. Prior to joining GM, Dr. 
Bender worked in the field of occupational and environmental medicine for nearly 
25 years, in leadership positions with the Campbell Soup Company. Owens- 
Coming, and Dupont. His activities have included clinical, executive, governing, 
and medical advisory roles in many national, international, and professional or- 
ganizations, and he recently served a four-year term on the Centers for Disease 
Control and Prevention's Advisory Committee. He received an M.D, degree from 
the University of Alabama's School of Medicine, a Ph.D. in environmental sci- 
ences from Drexel University, and is certified as a specialist in occupational and 
environmental medicine. 
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EULA BINGHAM (lOM), Ph.D., is professor of environmental health at the 
University of Cineinnati College of Medieine. Her interests inelude risk assess- 
ment, regulatory toxieology, environmental carcinogenesis, and occupational 
health surveillance. She was assistant secretary for labor for Occupational Safety 
and Health (OSHA) from 1977-1981. Throughout her career. Dr. Bingham has 
served on numerous national and international advisory groups, including advi- 
sory committees of the Food and Drug Administration, Department of Labor, Na- 
tional Institute for Occupational Safety and Health, National Institutes of Health, 
Natural Resources Defense Council, and the International Agency for Research 
on Cancer. The committees addressed issues concerning research needs in health 
risk assessment and the potential health effects of environmental exposure to 
chemicals. Dr. Bingham has a Ph.D. from the University of Cincinnati in zoology 
(physiology), ecology, and biochemistry. She is an lOM member who has served 
on a number of committees of The National Academies. 

JAMES E. CONE, M.D., M.P.H., is the medical director of the World Trade 
Center Health Registry, in the Division of Epidemiology at the New York City 
Department of Health and Mental Hygiene. He previously directed programs in 
environmental and occupational medicine at the city and state levels, and in the 
university environment, including over 20 years of experience teaching, perform- 
ing clinical and apphed epidemiological research and practicing clinical occupa- 
tional and internal medicine. Dr. Cone is currently the immediate past chairperson 
of the Occupational Health and Safety Section of the American Public Health As- 
sociation. His research interests are in occupational and environmental health, 
particularly toxic effects of exposures in the indoor environment, and health and 
mental health consequences of chemical and other environmental disasters, spills, 
and explosions, fires, and building collapses. He holds an M.D. degree from the 
University of California at San Francisco and an M.P.H. from the University of 
California al Berkeley. Other training in occupational and internal medicine in- 
cludes an internship and residencies at the Cook County Hospital in Chicago and 
the Worcester Memorial Hospital in Massachusetts, as well as a residency in epi- 
demic intelligence service training/occupational medieine in the Hazard Evalua- 
tion System and Information Service Branch of the Division of Surveillance, Haz- 
ard Evaluations and Field Studies at the Centers for Disease Control, National In- 
stitute for Occupational Safety and Health in Cincinnati, Ohio. 

MONICA GAUGHAN, Ph.D., M.P.A., is an assistant professor of health policy 
and management in the University of Georgia's College of Public Health. Her re- 
search interests focus on scientific and technical careers, research evaluation, sci- 
ence policy, health disparities, and reproductive health. Her research is currently 
supported by an early CAREER award from the National Science Foundafion. 
She has been a faculty member at Oglethorpe University, where she served 
as director of the Rich Foundation Urban Leadership Program, and the Georgia 
Institute of Technology. She served as a presidential management intern in the 
U.S. Department of Health and Human Services from 1992 to 1994, where she 
had assignments in the Offices of the Assistant Secretary for Health, the Assistant 
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Secretary for Legislation, the Health Resources and Services Administration, and 
the National Institutes of Health. She also has experience working in the commu- 
nity mental health system as a case manager and substance abuse counselor. Dr. 
Gaughan earned her Ph.D. in sociology from the University of North Carolina at 
Chapel Hill and her M.P.A. from the Maxwell School of Citizenship and Public 
Affairs, Syracuse University. 

CLARION JOHNSON, M.D., is currently the global medical director, Medicine 
and Occupational Health, for Exxon Mobil Corporation, which delivers industrial 
hygiene, drug testing and health promotion services to over 80,000 ExxonMobil 
and affiliate employees who arc engaged in exploration and production in a num- 
ber of challenging environments worldwide. Dr. Johnson received his M.D. from 
Yale University School of Medicine and is board certified in internal medicine, 
cardiology, and occupational medicine. In addition to a cardiology fellowship. Dr. 
Johnson did a military/basic science fellowship at Walter Reed Army Institute of 
Research, followed by two years as a postdoctoral student in the field of micro- 
wave research. He is currently a board member to the Milbank Memorial Fund 
and National Business Group on Health, and was former chairman of the Virginia 
Health Care Foundation. He has published a variety of articles in various fields. 

FRANKLIN E. MIRER, Ph.D., has recently assumed a faculty position at the 
Department of Environmental and Occupational Health Sciences at Hunter Col- 
lege after 25 years as Director of the Health and Safety Department for the United 
Automobile, Aerospace and Agricultural Implement Workers of America (UAW). 
He has a Ph.D. in physical organic chemistry from Harvard University and is a 
toxicologist and certified industrial hygienist. His primary scientific interest is 
exposure and risk assessment in the occupational environment. Dr. Mirer cur- 
rently serves on the National Academics Committee to Review NIOSH Research 
Programs, and in the past served on committees on Institutional Means for Risk 
Assessment, Risk Assessment Methodology, and the Review of the Health Effects 
Institute. He has testified before House and Senate committees on occupational 
safety and health matters and before OSHA on many health and safety standards. 
Dr. Mirer was inducted into the National Safety Council's Health and Safety Hall 
of Fame and is a fellow of the Collegium Ramazzini and the American Industrial 
Hygiene Association. 
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BARBARA SILVERSTEIN, Ph.D., M.P.H., is the research director of the 
Washington State Department of Labor & Industries Safety and Health Assess- 
ment and Research for Prevention (SHARP) Program. She has worked on ergo- 
nomics-related issues at OSHA, the University of Michigan Center for Ergonom- 
ics, the Finnish Institute of Occupational Health, and the California Department of 
Health Services. Her major areas of research have been identification and control 
of work-related musculoskeletal disorders, comparison of surveillance methods 
and intervention studies to control these disorders. She has conducted field re- 
search in a number of industries, including electronics, meat, poultry and fish 
processing, newspaper publishing, appliance manufacturing, medical equipment 
manufacturing, office work environment, pulp and paper mills, aluminum mills, 
saw mills, construction, automotive manufacturing, and nursing homes. She 
serves on a number of national and international commissions and editorial boards 
regarding occupational safety and health. She received her M.S. from the Univer- 
sity of California San Francisco, her M.P.H. in epidemiology and environmental 
and industrial health from the University of Michigan, and her Ph.D. in epidemi- 
ologic science from the University of Michigan. 

ROSEMARY K. SOKAS, M.D., M.Sc, M.O.H., is professor of environmental 
and occupational health sciences at the University of Illinois at the Chicago 
School of Public Health, and director of the Illinois Public Health Research Fel- 
lowship Program. Her research interests include applied intervention effectiveness 
studies targeting occupational safety and health needs of vulnerable working 
populations. She previously served on the faculties of the University of Pennsyl- 
vania School of Medicine and George Washington University, directed OSHA's 
Office of Occupational Medicine, and served as associate director for science at 
NIOSH. Dr. Sokas is currently a member of The National Academies' Committee 
on the Review of NIOSH Research Programs, and has served on the Committee 
on Persian Gulf Syndrome Comprehensive Clinical Evaluation Program and the 
Committee to Review the Worker and Public Health Activities Program Adminis- 
tered by the Department of Energy and the Department of Health and Human 
Services. She has an M.D. from the Boston University School of Medicine, and 
an M.Se. and M.O.H. from the Harvard School of Public Health (occupational 
physiology and occupational health, respectively). 

MICHAEL J. WRIGHT, M.S., is the director of Hcahh, Safety and Environ- 
ment for the United Steelworkers, a labor union representing American and Cana- 
dian workers in the steel, paper, oil, chemical, rubber, forestry, mining, and other 
industries. He has held this position since 1983. His areas of expertise include risk 
communication, occupational and environmental health, global health and safety, 
and industrial hygiene. Mr. Wright has considerable experience with the National 
Institute for Occupational Safety and Health's Health Hazard Evaluation Program, 
having requested and participated in numerous health hazard evaluafions on be- 
half of workers he represents. He was a member of the National Academies 
Committee on Industrial Compefitiveness and Environmental Protection and the 
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National Advisory Committee on Occupational Safety and Health. Mr. Wright 
holds an M.S. degree from the Harvard School of PubUc Health. 



NRC Staff 

SAMMANTHA L. MAGSINO is a program officer with the National Research 
Council's Board on Earth Sciences and Resources. She was previously a geologist 
with the Washington State geologic survey, where she produced earthquake haz- 
ard maps and served as a technical advisor to the state on volcanic hazards. Previ- 
ously, she was the science coordinator for a National Science Foundation facility 
at The University of Texas at Austin conducting aerogcophysicai surveys in Ant- 
arctica, and has worked for the Center for Nuclear Waste Regulatory Analyses at 
the Southwest Research Institute conducting geophysical investigations near the 
proposed Yucca Mountain Nuclear Waste Facility in support of volcanic hazard 
assessment. She holds an M.S. degree in geology from Florida International Uni- 
versity. 

SUSAN R. MCCUTCHEN is a senior program associate at the Board on Health 
Sciences Policy, She has been on staff at The National Academies for 27 years 
and has worked in several institutional divisions and with many different boards, 
committees, and panels within those units. The studies in which she has partici- 
pated have addressed a broad range of subjects and focused on a variety of issues 
related to science and technology for international development, technology 
transfer, aeronautics and the U.S. space program, natural disaster mitigation. U.S. 
education policy and science curricula, needle exchange for the prevention of HIV 
transmission, the scientific merit of the polygraph, human factors/engineering, 
research ethics, disability compensation programs, health hazard evaluation, and 
medical and public health preparedness for catastrophic events. She has assisted 
in the production of more than 50 publications. Ms. McCutchen has a B.A. in 
French, with minors in Italian and Spanish, from Ohio's Miami University, and an 
MA in French, with a minor in English, from Kent State University. 

TONYA E. FONG YEE is a senior program assistant with the Board on Earth 
Science and Resources. She received her B.S. in business administration, with a 
focus on marketing, from the University of Florida in Gainesville, Florida. Before 
coming to The National Academics, she interned at the Substance Abuse and 
Mental Health Services Administration, working on the Safe Schools/Healthy 
Students Initiative. 



PrepubHcation Version - Subject to Further Editorial Revision 



